Configuration of the Actor and Critic Network of the Deep Reinforcement Learning controller for Multi-Energy Storage System

Páramo-Balsa, Paula; Gonzalez-Longatt, Francisco; Acosta Montalvo, Martha N.; Rueda, José L.; Palensky, P.; Sanchez, Francisco; Roldan-Fernandez, JM; Burgos-Payán, Manuel

doi:10.1109/GPECOM55404.2022.9815793

Configuration of the Actor and Critic Network of the Deep Reinforcement Learning controller for Multi-Energy Storage System

Conference paper (2022)

Authors

Paula Páramo-Balsa University of Seville

Francisco Gonzalez-Longatt University of South-Eastern Norway

Martha N. Acosta Montalvo University of South-Eastern Norway

José L. Rueda Intelligent Electrical Power Grids -

P. Palensky Intelligent Electrical Power Grids -

Francisco Sanchez Loughborough University

JM Roldan-Fernandez University of Seville

Manuel Burgos-Payán University of Seville

Research Group

Intelligent Electrical Power Grids () (TU Delft)

DOI: https://doi.org/10.1109/GPECOM55404.2022.9815793

Parallel computing Deep reinforcement learning Energy storage systems Actor-network Critic network Enhanced frequency response

To reference this document use:

http://resolver.tudelft.nl/uuid:5ddd951e-8708-4961-b95e-12b0c5427d03

More Info

expand_more

Published Date

2022

Language

English

Reuse Rights

Other than for strictly personal use, it is not permitted to download, forward or distribute the text or part of it, without the consent of the author(s) and/or copyright holder(s), unless the work is under an open content license such as Creative Commons.

Faculty

Electrical Engineering, Mathematics and Computer Science

Department

Electrical Sustainable Energy

Research Group

Intelligent Electrical Power Grids

Abstract

The computational burden and the time required to train a deep reinforcement learning (DRL) can be appreciable, especially for the particular case of a DRL control used for frequency control of multi-electrical energy storage (MEESS). This paper presents an assessment of four training configurations of the actor and critic network to determine the configuration training that produces the lower computational time, considering the specific case of frequency control of MEESS. The training configuration cases are defined considering two processing units: CPU and GPU and are evaluated considering serial and parallel computing using MATLAB® 2020b Parallel Computing Toolbox. The agent used for this assessment is the Deep Deterministic Policy Gradient (DDPG) agent. The environment represents the dynamic model to provide enhanced frequency response to the power system by controlling the state of charge of energy storage systems. Simulation results demonstrated that the best configuration to reduce the computational time is training both actor and critic network on CPU using parallel computing.

Files

Configuration_of_the_Actor_and... (pdf)

(pdf | 1.09 Mb)

- Embargo expired in 11-01-2023

Unknown license