This paper investigates the deep reinforcement learning based secure control problem for cyber–physical systems (CPS) under false data injection attacks. We describe the CPS under attacks as a Markov decision process (MDP), based on which the secure controller design for CPS under attacks is formulated as an action policy learning using data. Rendering the soft actor–critic learning algorithm, a Lyapunov-based soft actor–critic learning algorithm is proposed to offline train a secure policy for CPS under attacks. Different from the existing results, not only the convergence of the learning algorithm but the stability of the system using the learned policy is proved, which is quite important for security and stability-critical applications. Finally, both a satellite attitude control system and a robot arm system are used to show the effectiveness of the proposed scheme, and comparisons between the proposed learning algorithm and the classical PD controller are also provided to demonstrate the advantages of the control algorithm designed in this paper.

This article investigates the secure control problem for cyber-physical systems when the malicious data are injected into the cyber realm, which directly connects to the actuators. Based on moving target defense (MTD) and reinforcement learning, we propose a novel proactive and reactive defense control scheme. First, the system (A,B) is modeled as a switching system consisting of several controllable pairs (A,Bl) to facilitate the construction of the MTD control scheme. The controllable pairs (A,Bl) can be altered to update system dynamics under certain unpredictable switching probabilities for each subsystem, which can prevent the adversaries from effective attacks. Second, both attack detection and isolation schemes are designed to accurately locate and exclude the compromised actuators from a switching sequence. Third, a reinforcement learning algorithm based on the zero-sum game theory is proposed to design the defense control scheme when there exist no controllable subsystems to switch. To demonstrate the effectiveness of the defense control scheme, a three-tank system under unknown cyber attacks is illustrated.

This paper investigates the problem of optimal tracking control for cyber-physical systems (CPS) when the cyber realm is attacked by denial-of-service (DoS) attacks which can prevent the control signal transmitting to the actuator. Attention is focused on how to design the optimal tracking control scheme without using the system dynamics and analyze the impact of DoS attacks on tracking performance. First, a Riccati equation for the augmented system including the system model and the reference model is derived under the framework of dynamic programming. The existence and uniqueness of its solution are proved. Second, the impact of the successful DoS attack probability on tracking performance is analyzed. A critical value of the probability is given, beyond which the solution to the Riccati equation cannot converge. The tracking controller cannot be designed. Third, reinforcement learning is introduced to design the optimal tracking control schemes, in which the system dynamics are not necessary to be known. Finally, both a dc motor and an F16 aircraft are used to evaluate the proposed control schemes in this paper.

This article investigates the zero-sum game-based secure control problem for cyber-physical systems (CPS) under the actuator false data injection attacks. The physical process is described as a linear time-invariant discrete-time model. Both the process noise and the measurement noise are addressed in the design process. An optimal Kalman filter is given to estimate the system states. The adversary and the defender are modeled as two players. Under the zero-sum game framework, an optimal infinite-horizon quadratic cost function is defined. Employing the dynamic programming approach, the optimal defending policy and the attack policy are derived. The convergence of the cost function is proved. Moreover, the critical attack probability is derived, beyond which the cost cannot be bounded. Finally, simulation results are provided to validate the proposed secure scheme.