Towards maximum efficiency in heat pump operation: Self-optimizing defrost initiation control using deep reinforcement learning

Air Source Heat Pumps (ASHPs) are a key technology in sustainable heating and cooling applications. Using air as heat source in cold climate conditions causes frost related performance degradation, and thus frequent defrosting is necessary. Typically, demand-based defrost initiation methods detect frost with sensors and initiate defrosting when a certain threshold value is reached. However, the performance of these methods is limited to the quality of the threshold value. State-of-the-art applications often assume a constant threshold value that is independent of operating condition. Further, the threshold value is usually determined heuristically based on simplified rules. To overcome these limitations, this study proposes a self-optimizing defrost initiation controller that utilizes deep reinforcement learning (RL). The RL controller autonomously extracts an efficient defrosting strategy under dynamic frosting conditions through a trial-and-error process. The proposed controller is designed to maximize heat pump performance and learns to detect frost using standard sensors of the refrigerant cycle. In a 31-day simulation study, the developed algorithm outperforms time-controlled and demand-controlled methods, resulting in an average efficiency improvement of 12.3% and 6.2%, respectively. Despite the promising results, open research questions must be addressed before RL can be applied to real heat pumps.