Poor Trader Dashboard 2026
Posts by Collection
portfolio
publications
Energy Efficient Operation Optimization of Building Air-conditioners via Simulator-assisted Asynchronous Reinforcement Learning
Abstract: At present, buildings are the major energy consumers in most developed and developing countries. Energy efficiency is one of the primary objectives of today’s building projects, especially in the context of carbon-peak and carbon-neutral. In buildings, air-conditioning systems account for nearly half of total building energy consumption. Optimal control and operation can significantly improve air-conditioning’s energy efficiency but the key challenge is the complex dynamics of air-conditioning systems. This study focuses the energy efficient operation of air-conditioners in office buildings, and proposes a simulation-assisted reinforcement learning (RL) method to develop energy efficient operation strategies. This study uses a whole building energy simulation model to build the environmental simulator for the air-conditioning system, and uses asynchronous RL algorithm to learn energy efficient operation strategies. Energy saving and Setpoint Not Met is used as the optimization objective as well as the criteria for evaluating the performance of the RL operation strategies. A validation simulator with varied weather conditions is also built to validate the robustness of reinforcement learning. The results show that, compared to common rule-based control strategy, 16.1% energy saving with better thermal comfort can be achieved by the RL operation strategy. In addition, the results also show the RL operation strategy has a certain level of robustness.
Recommended citation: Zhang, W., & Zhang, Z. (2022). Energy Efficient Operation Optimization of Building Air-conditioners via Simulator-assisted Asynchronous Reinforcement Learning. IOP Conference Series: Earth and Environmental Science, 1048(1), 012006. doi:10.1088/1755-1315/1048/1/012006
Reinforcement Learning in Building Controls: a Comparative Study of Algorithms considering Model Availability and Policy Representation
Abstract: Reinforcement Learning (RL) has presented considerable potential as an advanced control technique in building controls to enable buildings operating more energy-efficient. As various types of RL algorithms have been studied on their performance of building controls, benchmarking these algorithms across the entire spectrum of features is essential to provide an overview and deepen the understanding of RL applications. Therefore, this study aims to compare and analyze the effectiveness of various RL algorithms, encompassing the entire RL categories featured by value-based, policy gradient, actor-critic and model-based RL considering model availability and policy representation. To provide a comprehensive analysis, in addition to the control performance quantified by the cumulative rewards based on the cost function of RL, data demand and robustness of hyperparameter tuning were investigated. The open-source Gym-Eplus framework was selected as the virtual environment to train and test different RL agents. The results showed that both model-free and model-based RL agents outperformed the baseline rule-based control in terms of energy consumption and thermal comfort, and RL agents were capable of evaluating both short-term and long-term rewards to achieve adaptive control optimization continuously along with the online control process. Model-based RL agent improved the data sampling efficiency but presented a relatively sacrificed control performance during the tested summer days.
Recommended citation: Wu, Z., Zhang, W., Tang, R., Wang, H., & Korolija, I. (2024). Reinforcement learning in building controls: A comparative study of algorithms considering model availability and policy representation. Journal of Building Engineering, 90, 109497. doi:10.1016/j.jobe.2024.109497
Recommender systems and reinforcement learning for human-building interaction and context aware support: A text mining-driven review of scientific literature
Abstract: The indoor environment significantly impacts human health and well-being; enhancing health and reducing energy consumption in these settings is a central research focus. With the advancement of Information and Communication Technology (ICT), recommendation systems and reinforcement learning (RL) have emerged as promising approaches to induce behavioral changes to improve the indoor environment and energy efficiency of buildings. This study aims to employ text mining and Natural Language Processing (NLP) techniques to thoroughly examine the connections among these approaches in the context of human-building interaction and occupant context-aware support. The study analyzed 27,595 articles from the ScienceDirect database, revealing extensive use of recommendation systems and RL for space optimization, location recommendations, and personalized control suggestions. Although these systems are broadly applied to specific content, their use in optimizing indoor environments and energy efficiency remains limited. This gap likely arises from the need for interdisciplinary knowledge and extensive sensor data. Traditional recommendation algorithms, including collaborative filtering, content-based and knowledge-based methods, are commonly employed. However, the more complex challenges of optimizing indoor conditions and energy efficiency often depend on sophisticated machine learning (ML) techniques like reinforcement and deep learning. Furthermore, this review underscores the vast potential for expanding recommender systems and RL applications in buildings and indoor environments. Fields ripe for innovation include predictive maintenance, building-related product recommendation, and optimization of environments tailored for specific needs, such as sleep and productivity enhancements based on user feedback. The study also notes the limitations of the method in capturing subtle academic nuances. Future improvements could involve integrating and fine-tuning pre-trained language models to better interpret complex texts.
Recommended citation: Zhang, W., Quintana, M., & Miller, C. (2025). Recommender systems and reinforcement learning for human-building interaction and context aware support: A text mining-driven review of scientific literature. Energy and Buildings, 329, 115247. doi:10.1016/j.enbuild.2024.115247
Bedroom ventilation and air quality during sleep: Insights from a pilot field study in Singapore
Abstract: This study investigated bedroom ventilation and air quality in Singapore’s tropical climate. Comfort parameters along with multiple air pollutants were measured in seven bedrooms, each for approximately one month. The median bedroom ventilation rate with the interquartile range was 6.2 [3.4-23.1] L/s/p during sleep, and the CO2 concentration was 727 [516-1232] ppm. Both often did not meet the established standard requirements. TVOCs, formaldehyde, and PM2.5 levels remained below existing standards. The median temperature and relative humidity were 28.9 [27.9-29.7] °C and 77 [67-80] %, respectively. The bedroom light intensity during sleep was below 50 lux. Opening windows and doors improved ventilation and reduced CO₂ concentration but increased sound pressure level, frequently surpassing the standard limit. These preliminary findings indicate that existing bedrooms in Singapore might be insufficiently ventilated, and there is a potential need to balance fresh air supply with thermal comfort and noise control for better sleep and well-being.
Recommended citation: Fan, X., Zhang, W., Mani, R., Miller, C., Parkinson, T., Lo, J. C.-Y., Lee, J. K. W., Zhang, H., & Schiavon, S. (2025). Bedroom ventilation and air quality during sleep: Insights from a pilot field study in Singapore. Healthy Building Conference 2025.
Wrist to Rest: A pilot study to characterize sleep habits and bedroom environments using scalable watch-based microsurveys
Abstract: Rising temperatures from climate change and urban development can disrupt sleep and discourage outdoor activities, especially in already hot and humid places. The high prevalence of sleep deprivation in Singapore, linked to health and economic consequences, emphasizes the importance of addressing sleep quality in working-age adults. This paper outlines the methodology and preliminary results of the field-based portion of a study on understanding and mitigating the impacts of urban heat on sleep and physical activity. The study uses an open-source data collection framework to acquire data from participants and their bedrooms using a smartwatch application and environmental sensors. Most of the participants (70%) did not report a preference to change the thermal environment of the bedroom, while 29% preferred a cooler environment. The most frequently reported reasons for sleep disruptions were bathroom visits (34%) and being too hot (21%). These preliminary insights highlight the value of combining subjective and sensor-based data and set the stage for a broader data collection across 150 participants, which will enable deeper analyses of how bedroom environments and cooling behaviors impact sleep in tropical urban contexts.
Recommended citation: Miller, C., Chua, Y. X., Frei, M., Zhang, W., Kyaw, M. M., Seah, J. X., Fan, X., Parkinson, T., Zhang, H., Lo, J., Lee, J., & Schiavon, S. (2025). Wrist to Rest: A pilot study to characterize sleep habits and bedroom environments using scalable watch-based microsurveys. Journal of Physics: Conference Series, 3140(7), 072005. doi:10.1088/1742-6596/3140/7/072005
Poster Abstract: Field evaluation of body thermoregulation-based dynamic bedroom air temperature control to improve sleep
Abstract: Sleep is essential for human health and well-being. Previous lab-based studies have shown that dynamically adjusting bedroom temperature according to body thermoregulation can improve sleep quality and thermal satisfaction. However, such strategies have not been tested in real-world settings, and there is a lack of practical solutions developed for residential buildings. In this work, we evaluate a body thermoregulation-based dynamic air conditioning (AC) control strategy designed for typical Singapore homes in a pilot field deployment. The preliminary findings suggest that the proposed strategy can be successfully implemented in real bedroom environments and may improve sleep outcomes.
Recommended citation: Zhang, W., Schiavon, S., & Miller, C. (2025). Field evaluation of body thermoregulation-based dynamic bedroom air temperature control to improve sleep. Proceedings of the 13th ACM International Conference on Systems for Energy-Efficient Buildings, Cities, and Transportation (BuildSys 2025). doi:10.1145/3736425.3772103
Just-in-Time Adaptive Interventions (JITAI) to Improve Indoor Air Quality in Sleep Environments: A Pilot Study
Abstract: Just-in-Time Adaptive Interventions (JITAIs) are used in mobile health to support self-management by delivering timely and personalized assistance. However, their effectiveness in prompting occupants to adjust their physical environment to improve indoor air quality (IAQ) remains unclear. As a pilot of a large-scale field campaign, we developed a smartwatch-based JITAI that nudges occupants toward IAQ-improving behaviors. Its effectiveness was evaluated through a micro-randomized field trial in Singapore (n = 12), with each participant completing 60 valid study days. The results show that ~90% of participants perceived the JITAI as helpful. Indicative window-opening events, used as a proxy for behavioral responses to JITAI prompts, were ~2.6 times more frequent on days with JITAI than without (15.8% vs. 6.2%), but the recommended 20-minute ventilation period was insufficient to improve IAQ. Future studies should improve the JITAI message content and incorporate feedback mechanisms that help users recognize the positive effects of their actions.
Recommended citation: Zhang, W., Frei, M., Chua, Y. X., Renard, M., Fan, X., Chong, A., Parkinson, T., Lee, J. K. W., Schiavon, S., & Miller, C. (2026). Just-in-Time Adaptive Interventions (JITAI) to Improve Indoor Air Quality in Sleep Environments: A Pilot Study. Indoor Air 2026.
Bedroom Nighttime Temperature, Humidity and Sleep Duration in Singapore Homes
Abstract: Warming nights are increasingly recognized as a determinant of sleep loss, yet most evidence relies on outdoor temperature proxies that may not reflect the indoor bedroom exposures that occupants can modify. We assessed how night-to-night variation in bedroom thermal environments relates to total sleep time in a longitudinal study of working adults in Singapore, using data from 33 participants over 1,460 nights (mean 44 nights/participant). Bedrooms were found to be consistently warm and humid, but within-person associations between bedroom temperature, humidity, Heat Index and nightly total sleep time were small and imprecise across model specifications. Preliminary analyses assessing non-linearity did not support a single indoor ‘warm-night’ threshold. This suggests that actionable indoor guidance for sleep may need to consider both thermal conditions and night-to-night behavioral responses (e.g., cooling and ventilation practices), rather than temperature alone, pending confirmation in the full dataset.
Recommended citation: Renard, M., Miller, C., Frei, M., Chua, Y. X., Tan, P. M. S., Kyaw, G. M. M., Zhang, W., Seah, J. X. T., Fan, X., Zhang, H., Parkinson, T., Lo, J. C.-Y., Lee, J. K. W., & Schiavon, S. (2026). Bedroom Nighttime Temperature, Humidity and Sleep Duration in Singapore Homes. Indoor Air 2026.
talks
Energy Efficient Operation Optimization of Building Air-conditioners via Simulator-assisted Asynchronous Reinforcement Learning
The presented study explored a reinforcement learning (RL)-based strategy for optimizing the energy-efficient operation of variable refrigerant flow (VRF) air-conditioners in office settings. The research addressed the significant energy consumption of air-conditioning systems, which account for a substantial proportion of building energy usage, and proposed an innovative solution using asynchronous reinforcement learning coupled with detailed building energy simulation models.
Invited Talk at China Academy of Building Research (CABR) on “The Opportunities and Challenges of Reinforcement Learning for Smart Building Control”
Reinforcement learning (RL) emerged as a transformative approach for optimizing smart building control systems, offering dynamic and adaptive solutions that significantly enhanced energy efficiency, occupant comfort, and operational sustainability. In this invited talk, the speaker delved into the evolving role of RL in the context of smart building technologies, emphasizing its potential to revolutionize how buildings responded to environmental conditions, occupancy patterns, and energy demands.
Text-Mining-Driven Review of Recommender Systems and Reinforcement Learning for Building Control and Occupant Interaction
This invited talk explored the application of text-mining techniques in conducting literature reviews, with a focus on the integration of recommender systems and reinforcement learning for smart building control and occupant interaction. Text-mining was presented as a powerful alternative to conventional literature review methods, enabling the analysis of large volumes of academic publications with improved efficiency and reduced subjectivity.
teaching
Mentorship
See you!