Reinforcement Learning in Building Controls: a Comparative Study of Algorithms considering Model Availability and Policy Representation

Abstract: Reinforcement Learning (RL) has presented considerable potential as an advanced control technique in building controls to enable buildings operating more energy-efficient. As various types of RL algorithms have been studied on their performance of building controls, benchmarking these algorithms across the entire spectrum of features is essential to provide an overview and deepen the understanding of RL applications. Therefore, this study aims to compare and analyze the effectiveness of various RL algorithms, encompassing the entire RL categories featured by value-based, policy gradient, actor-critic and model-based RL considering model availability and policy representation. To provide a comprehensive analysis, in addition to the control performance quantified by the cumulative rewards based on the cost function of RL, data demand and robustness of hyperparameter tuning were investigated. The open-source Gym-Eplus framework was selected as the virtual environment to train and test different RL agents. The results showed that both model-free and model-based RL agents outperformed the baseline rule-based control in terms of energy consumption and thermal comfort, and RL agents were capable of evaluating both short-term and long-term rewards to achieve adaptive control optimization continuously along with the online control process. Model-based RL agent improved the data sampling efficiency but presented a relatively sacrificed control performance during the tested summer days.

Keywords: Reinforcement LearningModel-free Reinforcement LearningModel-based Reinforcement LearningBuilding energy efficiencyControl optimization

Recommended citation: Z. Wu, W. Zhang, R. Tang, H. Wang, I. Korolija, Reinforcement Learning in Building Controls: a Comparative Study of Algorithms considering Model Availability and Policy Representation, Journal of Building Engineering, https://doi.org/10.1016/j.jobe.2024.109497. https://www.sciencedirect.com/science/article/pii/S2352710224010659?via%3Dihub