oa Self-learning Control Of Thermostatically Controlled Loads: Practical Implementation Of State Of The Art In Machine Learning.

Optimal control of thermostatically controlled loads such as air conditioning and hot water storage plays a pivotal role in the development of demand response. Demand response being an enabling technology in a society with an increased electrification and growing production from intrinsically stochastic renewable energy. Optimal control however, often relies on the availability of a system model in combination with an optimizer, a popular approaches being model predictive control. Building such a controller, is considered a cumbersome endeavor requiring custom expert knowledge, making large scale deployment of similar solutions challenging. To this end we propose to leverage on recent developments in machine learning, enabling a practical implementation of a model-free controller. This model-free controller interacts with the system within safety and comfort constraints and learns from this interaction to make near-optimal decisions. All of this within a limited convergence time on the order of 20-40 days. When successful, self-learning control allows for a large scale cost-effective deployment of demand response applications supporting a future with increased uncertainty in the energy production. To be more precise, recent results in the field of batch reinforcement learning and regression algorithms such as extremely randomized trees open the door for practical implementations. To support this, we intend in this work to show our most recent experimental results in implementing generic self-learning controllers for thermostatically controlled loads showing that indeed near optimal policies can be obtained within a limited time.