Research Group
The Center for Combined Smart Energy Systems (CoSES) investi-gates smart operation strategies for sustainable small-scale energy systems, like microgrids or districts. Our focus lies on the complex interaction of different subsystems, considering three layers: thermal energy supply, electrical energy supply, communication & computa-tion. The research of our young group covers the full spectrum from method development & algorithm implementation to simulative inves-tigations and experimental validation. For the latter, we operate and use a unique microgrid laboratory comprising five building emulators, which are connected by three grids – communication, heat and elec-tricity. The real-world analogy of the lab is a microgrid of five houses, four single-family houses (SFH) and one multi-family building (MFH). The Hardware-in-the-loop (HiL) philosophy we follow allows us to test developed control strategies safely in real-time in the lab. Simulation models built and run with common software, like Dymola complete the setup.
Background
On the thermal side we focus on innovative district heating and cool-ing networks of the 4th and 5th generation. A distinguishing feature of these networks are decentral prosumers that can feed-in or extract thermal energy from the network to optimize the power flows within the microgrid for maximum efficiency. This evokes bidirectional power and mass flows in the network, leading to new thermohydraulic chal-lenges for the network operation.
Research Topic 1: Reinforcement Learning-Enhanced Control of Bidirectional Substations in Thermal Grids.
Thermal substations hydraulically separate and thermally couple the network-side and the prosumer-side. With a rising penetration of prosumers the substations become the key element for network oper-ation and control the power exchange on a technical level. Pumps and valves are the actuators controlling the volume flows, while the heat exchanger determines the thermal behavior. The necessity to be able to switch between production and consumption mode, while fulfilling several technical objectives, increases the complexity and importance of substation control. Heuristic- and model-based controllers have been designed by the research group. However, the system behavior is highly nonlinear, different substations mutually influence each other over the network, physical models require high parametrization effort and the adaptivity to changed network settings is limited. Therefore, as a next step, a data-driven control approach shall be developed based on reinforcement learning using local measurements in the individual substation. The topic comprises the following tasks:
• Familiarization with the control system, i.e. bidirectional prosumer substations
• Familiarization with existing models of the control system in Dymola (Modelica)
• Literature review on preceding and related work
• Choosing a suitable RL approach and a development envi-ronment (e.g. Python, Matlab etc.)
• Setting up a toolchain for the interaction of simulation (meas-urement) and RL-controller-environment, data acquisition
• Setting up and training the RL controller
• Benchmarking the developed RL-controller against an existing controller
• Result processing, evaluation and presentation
• Preparation of a report describing the key aspects and findings of the conducted work
• Disseminating clean source code via an open-source reposito-ry and presenting the results to a scientific audience
Research Topic 2: Data-driven grid state estimation in 4th & 5th Gen District Heating and Cooling Systems using machine learning meth-ods
In conventional district heating networks, only few network-side measurements are taken, in particular at the hydraulic worst-point. Due to the simple functional principle and the steady operation of conventional networks, these few measurements are enough to oper-ate the network with central pumping stations in the network. With an increasing penetration of prosumers, the network operation is decen-tralized to the substations and becomes much more dynamic and complex due to the mutual influence of the control actions. This makes it far more complex to anticipate the emerging system states. Comprehensive knowledge on the thermohydraulic state in the net-work at different locations becomes essential for a stable and smart network operation. As sensor equipment is expensive and prone to failure, one way to go is the estimation of the network state based on few local measurements. Due to the complexity of the thermal and hydraulic intercoupling of central and decentral actuators, physical models go along with high computational complexity. Additionally, they require high parametrization effort an are limited in the adaptation to changing network settings. Therefore, a machine-learning based approach for the grid state estimation during live-operation shall be developed and tested. The topic comprises the following tasks:
• Familiarization with 4th and 5th Gen. District Heating and Cool-ing (DHC), i.e. the thermohydraulics behind it and refreshing knowledge on state estimation and machine learning methods
• Familiarization with existing DHC models in Dymola (Modeli-ca), which are to be used for data generation and for testing in live-operation
• Literature review on preceding and related work
• Choosing a suitable ML approach and a development envi-ronment (e.g. Python, Matlab etc.)
• Setting up a toolchain for the interaction of simulation envi-ronment and controller-environment
• Developing and training the state estimator
• Benchmarking the estimator against the ground truth of simu-lated values from models
• Result processing, evaluation and presentation
• Preparation of a report describing the key aspects and findings of the conducted work
• Disseminating clean source code via an open-source reposito-ry and presenting the results to a scientific audience