Advancements in aerial vehicles have presented new challenges for flight control system design. The disturbed airflow caused by rotors and flapping wings and the nonlinearity and uncertainty increased by morphing components impede the identification of a globally accurate model of these vehicles. Additionally, complex components can increase the risk of structural damage and faults and pursuing large wingspans and a high aspect ratio can result in unstable flight dynamics because of structural flexibility. Moreover, current control systems are limited in their ability to adapt to unanticipated or changing circumstances. Given these challenges, the aerospace community has turned to intelligent methods to enhance the autonomy of flight controllers. Reinforcement learning (RL) has been popular in recent years because it offers self-learning approaches that improve agent policies through interaction with the environment. Fruitful cross-fertilization of RL and control theory produced adaptive dynamic programming (ADP), which has been successfully applied to a wide range of nonlinear control systems, including aerial vehicles. ADP inherits dynamic programming’s optimality while enabling adaptability. Furthermore, with the facilitation of artificial neural networks (ANNs), ADP can handle complex control demands and demonstrate the capability of online learning. Consequently, the main research goal of this dissertation is: To improve the adaptability and online learning capability of flight control systems by designing nonlinear ADP approaches.@en

The current study presents an online iterative adaptive dynamic programming approach to resolve the zero-sum game (ZSG) for nonlinear continuous-time (CT) systems containing a partially unknown dynamic. The Hamilton-Jacobian-Issacs (HJI) equation is solved along the state trajectory according to the value function approximation and the policy improvement online. Relaxed dynamic programming is utilized to ensure the algorithm’s convergence. Model and costate networks were established to conduct the method. Computational simulations are performed to present the efficiency of the algorithm.

@en

In this paper, we establish an event-triggered intelligent control scheme with a single critic network, to cope with the optimal stabilization problem of nonlinear aeroelastic systems. The main contribution lies in the design of a novel triggering condition with input constraints, avoiding the Lipschitz assumption on the inverse hyperbolic tangent function. Based on an improved weight updating criterion that eliminates the requirement of initial admissible control, the control law is obtained approximately by online training of a single critic network. The Lyapunov stability and the Zeno phenomenon of the closed-loop system are analysed. The feasibility of the established algorithm is verified by applying it to an optimal stabilization task of a nonlinear aeroelastic system. The results reveal that the developed approach can handle input-constrained optimal control problems, with performance comparable to the time-based method that updates control inputs at each instant, while reducing the computational and communication's load.

@en

Morphing structures have acquired much attention in the aerospace community because they enable an aircraft to actively adapt its shape during flight, leading to fewer emissions and fuel consumption. Researchers have designed, manufactured, and tested a morphing wing named SmartX-Alpha, which can actively alleviate loads while achieving the optimal lift distribution. However, the widely existing mechanical imperfections can degrade the performance of the morphing wing and even lead to instabilities. To tackle these issues, this article proposes a vision-based adaptive control approach to actively compensate for mechanical imperfections. In this approach, an incremental model is constructed online to identify the system dynamics using servo commands and vision measurements, and then, nonlinear dynamic inversion control is applied based on the identified model. This data-driven control approach with visual feedback has been validated by real-world experiments on the SmartX-Alpha. The results demonstrate that the vision-based system combined with the proposed control methodology can actively compensate for mechanical imperfections with minimal adjustments to the actual system design. Compared to a controller that only uses a feedforward input-output mapping, this proposed approach improves the system performance and decreases the tracking errors by more than 62% despite disturbances. The results collectively demonstrate the effectiveness of the proposed control system, which sets a foundation for realizing morphing in next-generation aircraft.

@en

This paper develops an event-triggered optimal control method that can deal with asymmetric input constraints for nonlinear discrete-time systems. The implementation is based on an explainable global dual heuristic programming (XGDHP) technique. Different from traditional GDHP, the required derivatives of cost function in the proposed method are computed by explicit analytical calculations, which makes XGDHP more explainable. Besides, the challenge caused by the input constraints is overcome by the combination of a piece-wise utility function and a bounding layer of the actor network. Furthermore, an event-triggered mechanism is introduced to decrease the amount of computation, and the stability analysis is provided with fewer assumptions compared to most existing studies that investigate event-triggered discrete-time control using adaptive dynamic programming. Two simulation studies are carried out to demonstrate the applicability of the constructed approach. The results present that the developed event-triggered XGDHP algorithm can substantially save the computational load, while maintain comparable performance with the time-based approach.

@en

Linear Approximate Dynamic Programming (LADP) and Incremental Approximate Dynamic Programming (IADP) are Reinforcement Learning methods that seek to contribute to the field of Adaptive Flight Control. This paper assesses their performance and convergence, as well as the impact of sensor noise on policy convergence, online system identification, performance and control surface deflection. After summarising their theory and derivation with full state (FS) and output feedback (OPFB), they are implemented on the linearised longitudinal F16 model. In order to establish an objective performance comparison, their hyper-parameters were tuned with an evolutionary algorithm: Particle Swarm Optimisation (PSO). Results show that LADP and IADP have the same performance in the presence of FS feedback, whereas LADP outperforms IADP when only OPFB is available. Output noise causes LADP based on OPFB to diverge. In the case of IADP based on OPFB, sensor noise improves the performance due to a better exploration of the solution space. The present research aims at bridging the gap between the discussed ADP algorithms and real world systems.

@en

The scarcity of information regarding dynamics and full-state feedback increases the demand for a model-free control technique that can cope with partial observability. To deal with the absence of prior knowledge of system dynamics and perfect measurements, this paper develops a novel intelligent control scheme by combining global dual heuristic programming with an incremental model-based identifier. An augmented system consisting of the unknown nonlinear plant and unknown varying references is identified online using a locally linear regression technique. The actor–critic is implemented using artificial neural networks, and the actuator saturation constraint is addressed by exploiting a symmetrical sigmoid activation function in the output layer of the actor network. Numerical experiments are conducted by applying the proposed method to online adaptive optimal control tasks of an aerospace system. The results reveal that the developed method can deal with partial observability with performance comparable to the full-state feedback control, while outperforming the global model-based method in stability and adaptability.

@en

A novel adaptive dynamic programming method, called incremental model-based global dual heuristic programming, is proposed to generate a self-learning adaptive flight controller, in the absence of sufficient prior knowledge of system dynamics. An incremental technique is employed for online local dynamics identification, instead of the artificial neural networks commonly used in global dual heuristic programming, to enable a fast and precise learning. On the basis of the identified model, two neural networks are adopted to facilitate the implementation of the self-learning controller, by approximating the cost-to-go and the control policy, respectively. The required derivatives of cost-to-go are computed by explicit analytical calculations based on differential operations. Both methods are applied to an online attitude tracking control problem of a nonlinear aerospace system and the results show that the proposed method outperforms conventional global dual heuristic programming in tracking precision, online learning speed, robustness to different initial states and adaptability for fault-tolerant control problems.

@en

Optimal tracking is a widely researched control problem, but the unavailability of sufficient information referring to system dynamics brings challenges. In this paper, an optimal tracking control method is proposed for an unknown launch vehicle based on the global dual heuristic programming technique. The nonlinear system dynamics is identified by an offline trained neural network and a feedforward neuro-controller is developed to obtain the desired system input and to facilitate the execution of the feedback controller. By transforming the tracking control problem into a regulation problem, an iterative adaptive dynamic programming algorithm, subject to global dual heuristic programming with explicit analytical calculations, is utilized to deal with the newly built regulation problem. The simulation results demonstrate that the developed method can learn an effective control law for the given optimal tracking control tasks.

@en

Sufficient information about system dynamics and inner states is often unavailable to aerospace system controllers, which requires model-free and output feedback control techniques, respectively. This paper presents a novel self-learning control algorithm to deal with these two problems by combining the advantages of heuristic dynamic programming and incremental modeling. The system dynamics is completely unknown and only input/output data can be acquired. The controller identifies the local system models and learns control polices online both by tuning the weights of neural networks. The novel method has been applied to a multi-input multi-output nonlinear satellite attitude tracking control problem. The simulation results demonstrate that, compared with the conventional actor-critic-identifier-based heuristic dynamic programming algorithm with three networks, the proposed adaptive control algorithm improves online identification of the nonlinear system with respect to precision and speed of convergence, while maintaining similar performance compared to the full state feedback situation.

@en

This paper proposes a novel adaptive dynamic programming method, called Incremental model-based Global Dual Heuristic Programming, to generate a self-learning adaptive controller, in the absence of sufficient prior knowledge of system dynamics. An incremental technique is employed for online model identification, instead of the artificial neural networks commonly used in conventional Global Dual Heuristic Programming. The incremental model has the capability of tackling nonlinearity and uncertainty of the plant, but can also guarantee high precision of online identification without the requirement of offline training. On the basis of the identified model, two neural networks are adopted to facilitate the implementation of the self-learning controller, by approximating the cost-to-go and its derivatives and the control policy, respectively. Both methods are applied to a tracking control problem of a nonlinear aerospace system and the results show that the proposed method outperforms conventional Global Dual Heuristic Programming in online learning speed, tracking precision and robustness to variation of initial system states and network weights.

@en