As the world is currently actively trying to reduce the consumption of fossil fuels, large investments are done in renewable energy sources and ways are sought after to electrify fossil fuel-intensive sectors. In line with these developments, the number of electric vehicles requiring access to the electric power grid has exploded putting increased pressure on the grid. One way to decrease the congestion in the grid is to make use of smart charging schedules for electric vehicles, with the objective of reducing peak demand and preventing the overloading of cables and transformers while reducing the cost of charging for electric vehicle owners.

The recent increase in the availability of real-life data has allowed the in-depth study of smart charging dynamics on a large scale through modeling and simulation. Mathematical optimization is a method that is often used to generate smart charging plans in state-of-the-art smart charging applications. However, while mathematical optimization can be very effective, as the objective function expands and more parameters are taken into account, the optimization becomes more complex and therefore require faster and smarter optimization algorithms. In addition, the recent availability of a vast amount of real-life data sets has made efficient data handling more important.

Machine learning is known to be an effective way of introducing Artificial Intelligence to smart charging algorithms. The use of reinforcement learning algorithms, a subset of machine learning could help overcome the disadvantages of mathematical optimization as trained algorithms are generally fast when predicting outcomes and have the potential to be accurate at the same time.

Therefore, the purpose of this work is to research the feasibility and additional benefit of machine learning to mathematical optimization-based smart charging algorithms. This is done through the development of end-to-end Q-learning and Double Deep Q Network reinforcement learning smart charging algorithms. The performance of both algorithms is evaluated on three separate case studies as well as on multiple different random cases and is compared to the charging performance of the average rate charging and mixed-integer programming algorithm benchmarks.

As a result, it becomes clear that for individual case studies the Q-learning and Double Deep Q Network agent are able to find cheap charging moments while charging the vehicle to 100\% battery capacity without violating charging constraints. However, when testing the performance of the Q-learning and Double Deep Q Network agents it becomes clear that the average charging performance is significantly worse than using the method of mixed-integer programming as the algorithms do not learn to generalize well.

Finally, the advantages and disadvantages of replacing mixed-integer programming with reinforcement learning are discussed as well as some limitations and recommendations for future work and improvement are given.

With the current urge around the globe for switching to “green energy” and Renewable Energy Sources (RES) utilization, the electrification of the transportation sector has become a necessity. While the emerging Electric Vehicles (EVs) can provide several benefits to the power grid, such as voltage and frequency regulation and power quality improvement, the large and immense EV fleets penetration can have the exact opposite results, if no suitable charging method is utilized. Therefore, the uncontrollable EV charging, that is being utilized today, will not be capable of offering sustainable energy to the future EVs with respect to the power grid limits and hence, today’s research field has turned to optimized smart-charging techniques. However, most of these algorithms treat the problem of smart-charging deterministically, assuming 100% accurate prediction of the input data, while it has been shown that the optimality of results can be seriously deteriorated even under small prediction error.This thesis purpose is to address the impact and potential management of several uncertainties related with EV smart charging: PV Generation, Load Demand, arrival SOC, Arrival and Departure time of the EVs & (Frequency Containment Regulation) FCR Reserves provision uncertainties. The main handling technique, utilized in this thesis, is the Robust Optimization (RO) approach, taking advantage of its very lower computational expense and protection against the worst-case scenario of the uncertainty, with the combination of the Receding Horizon Optimization (RHO), that is already implemented in the Benchmark algorithm. After improving the Benchmark algorithm by providing it with prediction feature, this thesis proves the capability of improved robustly EV charging with the combination of RO-RHO approach with prediction, which reduces the economical (in terms of charging costs/income) and “customer satisfaction” (in terms of unfinished charging gap impacts) of all the uncertainties considered. A more robust and “realistic” FCR reserves provision model has been developed, as well. Last but not least, the well-known RO drawback of potential over-conservativeness, in other words high deterioration of optimality at expense of robustness, is address for every uncertainty.The results of the investigation provide valuable results about which uncertainty has the greater impact, is more robustly manageable or inflicts the highest “Price of Robustness”, regarding overconservativeness. For example, it has been found that the FCR reserves uncertainty inflicts the highest economical and charging gap impacts, however it is the most robustly manageable uncertainty as well, if RO and prediction feature are utilized. Moreover, while arrival SOC and Parking Time uncertainties affect highly the unfinished charging gap, the PV Generation and Load Demand have practically only economic impact, which is also lower than the other uncertainties, hence they are defined as the “uncertainties with the least impact”. Finally, there are 3 types of nodes studied: the “Home” Node, the “Semi-Public” Node & the “Public” Nodes. Observing their behavior during the different uncertainties’ study cases, interesting results have been found about their robust management and affection by the various uncertainties.

Although electric vehicles were first used as early as the 19th century, they have only recently gained popularity. By charging them with renewable sources, electric vehicles present a route to zero or low emission mobility. This is a significant reason for their recent popularity given the climate crisis. However, the integration of electric vehicles comes with a unique set of problems, such as over-loading of grid elements and under-voltages of nodes. The integration of photovoltaic generation is also another source of complexity. The focus of this thesis is to investigate the influence of electric vehicles and solar generation in low voltage power grids.
Different charging algorithms are analysed and compared to the uncontrolled charging scenario. The price based charging algorithm is a centralised control strategy. The other three algorithms - average rate charging, nodal voltage charging, and smart charging - are decentralised strategies. Each of these algorithms manipulate the charging power at different chargers in the grid according to their specific conditions.
The aforementioned algorithms are tested by performing simulations in three types of German distribution grids: rural, urban, and sub-urban. These simulations are carried out for High PV, Low PV, and No PV scenarios. In all three grids, there is neither over-loading of equipment nor under-voltage of nodes in the No PV case. The under-voltages are improved due to photo-voltaic generation. In the High PV case, excessive generation causes over-loading and over-voltages in the sub-urban grid. The possibility of using electric vehicles as dynamic storage elements has been explored. Based on the insights obtained from the grid simulations, the performance of different charging algorithms is analysed and compared in this thesis.

The charging of a rapidly increasing number of EVs places significant strain on electrical distribution grids. Controlled (’smart’) charging of EVs offers a solution to this problem by shifting the charging of vehicles to the most optimal moments in time. Smart charging is not a completely new research topic; it has a high Technology Readiness Level and some commercial implementations already exist. There is a large amount of literature available that compares the effect of controlled and uncontrolled EV charging on the distribution grid. However, most of this literature is based on pure software simulations using power flow analysis and mathematical models of EV charging behaviour. Experimentally validating these results would require a large number of EVs and a controllable distribution grid, which is costly and unpractical. Therefore, this thesis presents a compromise; a hardware-in-the-loop (HIL) testbed in which the distribution grid is software-based but the EV and EV Supply Equipment (EVSE) is hardware-based. In this report, the design and implementation of the testbed is discussed, as well as results from HIL simulations that compare the effect of controlled and uncontrolled EV charging in a distribution grid. The low-cost HIL testbed presented in this thesis is based upon an OPAL-RT OP5700 Digital Real-Time Simulator (DRTS) which runs a Newton-Raphson power flow analysis of a Dutch distribution grid. The DRTS also interfaces with all other hardware in the testbed. This includes a power amplifier which generates a three-phase grid at the voltage calculated by the power flow analysis. A commercially available EVSE is connected to this power amplifier and supplies power to an emulated EV which consists of communication hardware and two bidirectional back-to-back DC power supplies. These power supplies act as an AC load. Various communication protocols were implemented to exchange information between the different systems in a manner that closely represents commercial EVs. A smart charging algorithm which determines the optimal charging current setpoints in real-time, based on external factors like local load power consumption, solar irradiance and energy prices, was implemented into the testbed. The testbed has subsequently been used to study the effect of uncontrolled and controlled charging on the simulated distribution grid for a total of eight different scenarios. Of these scenarios, one consists of traditional uncontrolled charging and the other seven are with the smart charging algorithm activated. These include firstly a base case, then two scenarios with an inaccurate solar irradiance and local load forecast, a further two scenarios with non-ideal EV charging behaviour, and lastly two scenarios with respectively centralized and decentralized curtailment implemented. In all scenarios, one EV was implemented as hardware and three others as software, so that a comparison could be made. Even under non-ideal conditions, smart charging is found to reduce costs, grid overloads, and voltage deviations with respect to uncontrolled charging. The implemented centralized curtailment is shown to be more effective than decentralized and further reduces voltage deviations in the grid but at a slightly higher cost than without this curtailment. Lastly, based on observations made in the HIL simulation results, recommendations for future work and improvement of the smart charging algorithm are given.