New research from Columbia University’s Building Energy Research Laboratory examined a practical question facing the growing field of AI-based building controls: how accurate does a building simulator need to be in order to train effective reinforcement learning controllers? The study, led by Matthew Kerr and Dr. Bianca Howard, tested whether simplified building models can still produce reliable temperature control when deployed in real buildings. 
To explore this question, the researchers trained reinforcement learning agents using four different EnergyPlus simulation models of the same residential test house. Each model varied key thermal parameters, including insulation and air infiltration, resulting in heat loss coefficients that differed by roughly 30 percent from one another. The agents were trained offline using deep Q learning and then evaluated both in simulation and in a real test house operated by BERL.
The results showed that all of the agents, regardless of the accuracy of the model they were trained on, were able to maintain indoor temperatures within ±1°C of the desired setpoint in both simulated tests and real-world deployment. While the agents learned different control strategies depending on the training environment, their performance in maintaining comfort remained consistent. This finding challenges the assumption that highly detailed and precisely calibrated simulation models are always necessary for training effective control algorithms.
The study also highlighted important nuances. Differences between simulation and real-world performance revealed that the real building had greater thermal mass than the models assumed, which affected how quickly temperatures responded to heating. While this did not degrade performance for temperature control, the authors note that model accuracy may matter more for other tasks such as energy arbitrage or demand response.
Overall, the research suggests that for basic temperature control, reasonably informed assumptions about building construction and operation may be sufficient to train high-performing reinforcement learning controllers. This has significant implications for real-world deployment, as it could reduce the need for extensive historical data collection and detailed calibration, making intelligent HVAC control faster and more accessible to implement in practice.
