Scalable Adaptive AC Control in Real Sleep Environments: A Pilot Deployment with Lightweight Reinforcement Learning
This conference talk presented a deployable research prototype for reinforcement learning (RL) based bedroom air-conditioning (AC) control in real homes. Sleep thermal comfort is dynamic and guided by a shifting neutral thermal sensation, and the setpoint profiles required for comfortable sleep can vary across nights. While RL has shown promise for personalized AC control, no field-deployable RL research prototype had previously been reported for residential sleep environments.
The presentation first outlined why laboratory-derived sleep setpoint curves are difficult to translate into real bedrooms. Real homes involve more variable thermal conditions, less precise temperature control, and diverse occupant behaviors and preferences. Neutral setpoints are also not fixed, as they can change with physiological state, sleep stage, bedding, outdoor weather, and AC performance. The resulting research gap is the lack of a scalable field tool for studying occupant-centric control in real homes. The aim was therefore to develop a prototype that supports occupant-centered adaptive AC setpoint control in real bedrooms over time.
The system integrates wearable sensing, indoor and outdoor environmental monitoring, subjective feedback, remote AC control via Sensibo Sky, and a cloud-based RL agent. The control algorithm first clusters nights with similar weather and indoor conditions, then uses bandit Q-learning to select among setpoints around the estimated neutral temperature. Occupants can still adjust the temperature when uncomfortable, and the agent learns from the actual setpoint and next-day feedback. The prototype also supports micro-randomized trials, with each night assigned equally to a baseline habitual setpoint or an RL-predicted setpoint. Preliminary results from an ongoing 60-day field deployment (n = 6, 30 nights per condition) showed that the system can operate reliably in real bedrooms and learn participant-specific setpoints from occupant feedback and behavioral adjustments.
The talk concluded by situating this prototype as a research platform rather than a finished control product. Ongoing work includes a field evaluation of body thermoregulation-based dynamic bedroom air-temperature control using the same protocol. Future directions include improving usability and extensibility through reusable control logic and broader smart-home device integration, and converting the prototype into a standardized experimental platform for residential occupant-centric control research.
Slide Download: Access the presentation slides here.
Related publication: Scalable Adaptive AC Control in Real Sleep Environments