District heating systems (DHSs) have the potential to play a big part in the energy transition. The efficient operation of DHSs is therefore also an important subject of study. The operation of DHSs where combined heat and power (CHP) plants are used are particularly interesting, because CHPs can operate with high efficiency.

In this work, the operational optimization of DHSs with CHP plants is considered. Determining the optimal heat and electricity production for CHPs for multiple time steps into the future is a complex problem. Because of the heat storage capabilities in the network many solutions are feasible, but determining which solutions are infeasible because of constraint violations in the DHS involves computing time delays that depend on complex network dynamics.

In this work, the possibility of using an input convex neural network (ICNN) to learn the network dynamics of a DHS is explored. ICNNs have limitations on their learning capabilities, but theoretically allow for easier optimization. Experiments on the learning capabilities of ICNNs reveal that caution should be used when they are used to learn non-convex constraints, as the accuracy of the ICNN highly depends on how non-convex the function is. Experiments on the feasible space of supply temperatures to a small district heating network (DHN) reveal that although the network does not provide the same flexibility as heat storage tanks, still some flexibility in the operation can be found. This is due to the fact that water with a higher supply temperature is consumed by consumers at a slower pace and this increases the time delay between production and consumption. Supply temperatures that follow can then be lowered if the increased time delay causes this water to arrive when the heat demands are lower.

In the experiments it was found that this flexibility in operation translates to non-convex areas in the feasible space. When this space would be learned by an ICNN, this space would be made convex. How much of the flexibility would be removed by doing this is yet unknown and could be researched in future work. Other future work can be done on safely learning non-convex constraints with an ICNN.

As the world is currently actively trying to reduce the consumption of fossil fuels, large investments are done in renewable energy sources and ways are sought after to electrify fossil fuel-intensive sectors. In line with these developments, the number of electric vehicles requiring access to the electric power grid has exploded putting increased pressure on the grid. One way to decrease the congestion in the grid is to make use of smart charging schedules for electric vehicles, with the objective of reducing peak demand and preventing the overloading of cables and transformers while reducing the cost of charging for electric vehicle owners.

The recent increase in the availability of real-life data has allowed the in-depth study of smart charging dynamics on a large scale through modeling and simulation. Mathematical optimization is a method that is often used to generate smart charging plans in state-of-the-art smart charging applications. However, while mathematical optimization can be very effective, as the objective function expands and more parameters are taken into account, the optimization becomes more complex and therefore require faster and smarter optimization algorithms. In addition, the recent availability of a vast amount of real-life data sets has made efficient data handling more important.

Machine learning is known to be an effective way of introducing Artificial Intelligence to smart charging algorithms. The use of reinforcement learning algorithms, a subset of machine learning could help overcome the disadvantages of mathematical optimization as trained algorithms are generally fast when predicting outcomes and have the potential to be accurate at the same time.

Therefore, the purpose of this work is to research the feasibility and additional benefit of machine learning to mathematical optimization-based smart charging algorithms. This is done through the development of end-to-end Q-learning and Double Deep Q Network reinforcement learning smart charging algorithms. The performance of both algorithms is evaluated on three separate case studies as well as on multiple different random cases and is compared to the charging performance of the average rate charging and mixed-integer programming algorithm benchmarks.

As a result, it becomes clear that for individual case studies the Q-learning and Double Deep Q Network agent are able to find cheap charging moments while charging the vehicle to 100\% battery capacity without violating charging constraints. However, when testing the performance of the Q-learning and Double Deep Q Network agents it becomes clear that the average charging performance is significantly worse than using the method of mixed-integer programming as the algorithms do not learn to generalize well.

Finally, the advantages and disadvantages of replacing mixed-integer programming with reinforcement learning are discussed as well as some limitations and recommendations for future work and improvement are given.