In this article, we present a reinforcement learning-based scheme for secondary frequency control of lossy inverter-based microgrids. Compared with the existing methods in the literature, we relax the common restrictions on the system, i.e., being lossless, and the transmission lines and loads to have known constant impedances. The proposed secondary frequency control scheme does not require a priori information about system parameters and can achieve frequency synchronization within an ultimate bound in the presence of dominantly resistive and/or inductive line and load impedances, model parameter uncertainties, and time varying loads and disturbances. First, using Lyapunov theory, a feedback control is formulated based on the unknown dynamics of the microgrid. Next, a performance function is defined based on cumulative costs toward achieving convergence to the nominal frequency. The performance function is approximated by a critic neural network in real-time. An actor network is then simultaneously learning a parameterized approximation of the nonlinear dynamics and optimizing the approximated performance function obtained from the critic network. Furthermore, using the Lyapunov approach, the uniformly ultimate boundedness of the closed-loop frequency error dynamics and the networks' weight estimation errors are shown.

@en

We present a privacy-preserving distributed reinforcement learning-based control scheme to address the problem of frequency control and economic dispatch in power generation systems. The proposed control approach requires neither a priori system model knowledge nor the mathematical formulation of the generation cost functions. Due to not requiring the generation cost models, the control scheme is capable of dealing with scenarios in which the cost functions are hard to formulate and/or non-convex. Furthermore, it is privacy-preserving, i.e. none of the units in the network needs to communicate its cost function and/or control policy to its neighbors. To realize this, we propose an actor-critic algorithm with function approximation in which the actor step is performed individually by each unit with no need to infer the policies of others. Moreover, in the critic step each generation unit shares its estimate of the local measurements and the estimate of its cost function with the neighbors, and via performing a consensus algorithm, a consensual estimate is achieved. The performance of our proposed control scheme, in terms of minimizing the overall cost while persistently fulfilling the demand and fast reaction and convergence of our distributed algorithm, is demonstrated on a benchmark case study.

@en

In this paper, we present a reinforcement learning control scheme for optimal frequency synchronization in a lossy inverter-based microgrid. Compared to the existing methods in the literature, we relax the restrictions on the system, i.e. being a lossless microgrid, and the transmission lines and loads to have constant impedances. The proposed control scheme does not require a priori information about system parameters and can achieve frequency synchronization in the presence of dominantly resistive and/or inductive line and load impedances, model parameter uncertainties, time varying loads and disturbances. First, using Lyapunov theory a feedback control is formulated based on the unknown dynamics of the microgrid. Next, a performance function is defined based on cumulative rewards towards achieving convergence to the nominal frequency. The performance function is approximated by a critic neural network in real-time. An actor network is then simultaneously learning a parameterized approximation of the nonlinear dynamics and optimizing the approximated performance function obtained from the critic network. The performance of our control scheme is validated via simulation on a lossy microgrid case study in the presence of disturbances.

@en

This paper presents a secondary voltage control scheme for microgrids based on the port-Hamiltonian modeling framework. The proposed secondary controller compensates the deviations of voltage amplitudes from their nominal values using the concept of energy shaping, which is the essence of passivity-based control in port-Hamiltonian systems. We shape the energy function and define a new Hamiltonian function such that the new potential energy function has a strict local minimum at the desired equilibrium point. Next, a feedback control is designed such that the closed-loop system preserves the port-Hamiltonian structure. The Hamiltonian in this case is the sum of the plant and the controllers energy functions. The stability analysis is performed and sufficient conditions on the controller gains to achieve voltage regulation are derived. The effectiveness of the proposed control methodology is evaluated using simulation for a benchmark microgrid system@en