Seasonal thermal energy storage systems (STESSs) can shift the delivery of renewable energy sources and mitigate their uncertainty problems. However, to maximize the operational profit of STESSs and ensure their long-term profitability, control strategies that allow them to trade on wholesale electricity markets are required. While control strategies for STESSs have been proposed before, none of them addressed electricity market interaction and trading. In particular, due to the seasonal nature of STESSs, accounting for the long-term uncertainty in electricity prices has been very challenging. In this article, we develop the first control algorithms to control STESSs when interacting with different wholesale electricity markets. As different control solutions have different merits, we propose solutions based on model predictive control and solutions based on reinforcement learning. We show that this is critical since different markets require different control strategies: MPC strategies are better for day-ahead markets due to the flexibility of MPC, whereas reinforcement learning (RL) strategies are better for real-time markets because of fast computation times and better risk modeling. To study the proposed algorithms in a real-life setup, we consider a real STESS interacting with the day-ahead and imbalance markets in The Netherlands and Belgium. Based on the obtained results, we show that: 1) the developed controllers successfully maximize the profits of STESSs due to market trading and 2) the developed control strategies make STESSs important players in the energy transition: by optimally controlling STESSs and reacting to imbalances, STESSs help to reduce grid imbalances.

Due to the increasing integration of renewable sources in the electrical grid, electricity generation is expected to become more uncertain. In this context, seasonal thermal energy storage systems (STESSs) are key to shift the delivery of renewable energy sources and tackle their uncertainty problems. In this paper, we propose an optimal controller for STESSs that, using reinforcement learning, builds bidding functions for the day-ahead market. In detail, considering that there is an uncertain energy demand that the STESS has to satisfy, the controller buys energy in the day-ahead market so that the uncertain demand is satisfied while the profits are maximized. Since prices are low during periods of large renewable energy generation (and vice versa), maximizing the profit of a STESS indirectly shifts the delivery of renewable energy to periods of high energy demand while reducing their uncertainty problems. To evaluate the proposed algorithm, we consider a real STESS providing different yearly-demand levels; then, we compare the performance of the controller to the theoretical upper bound, i.e. the optimal cost of buying energy given perfect knowledge of the demand and prices. The results indicate that the proposed controller performs reasonably well: despite the large uncertainty in prices and demand, the proposed controller obtains 70%-50% of the maximum gains given by the theoretical bound.