This paper provides a convergence and stability analysis of the incremental value iteration algorithm under the influence of various errors. Incremental control is firstly used to linearize the continuous-time nonlinear system, recursive least squares (RLS) identification is then introduced to identify the incremental model online. Based on the incremental model, the value iteration algorithm is used to design an optimal adaptive controller, with an analytical optimal control law. Moreover, the convergence of the developed incremental value iteration algorithm is proved. The stability of the controller is analyzed using Lyapunov stability theory. Finally, a flight control simulation verifies the robustness of the controller to various initial conditions, as well as adaptation to actuator faults.

To reduce the impact of aviation on the environment, technological innovations, such as the Flying-V are required. The Flying-V is a proposed commercial flying wing, which uses the Airbus A350-900 as reference aircraft. In this work, a Flight Control system for the Flying-V is proposed with a longitudinal C ^* control law, and a Rate Control Attitude Hold roll control law. This Flight Control System also includes a Flight Envelope Protection law to prevent reaching angles of attack higher than 30 degrees, where the Flying-V becomes statically unstable. The FEP also prevents the Flying-V from reaching load factors above 2.5 and limits the roll angle. The control laws are tuned to be within level 1 handling qualities in the selected approach and cruise conditions, with the presence of sensor dynamics, and a digital control system. Robustness for aerodynamic uncertainties is also shown. Finally, it is shown that the FEP is able to prevent the angle of attack from becoming too large.

This paper investigates the performance of an autonomous navigation system to navigate a spacecraft in the proximity of a binary asteroid system using optical and laser ranging measurements. The knowledge about the binary asteroid is limited to its orbital parameters and ellipsoid shape models. The accelerometer bias random walk is included in the estimation process. Over a four-hour landing maneuver starting from 6770 m altitude and ending at 550 m, the mean position estimation uncertainty is 41.6 m (3). It is shown that the navigation accuracy is sensitive to the Sun phase angle, the irregularity of the asteroid shape, and the goodness of fit of the ellipsoid shape model. The paper demonstrates that the navigation system is robust to large errors in the initialization of the extended Kalman filter state. The impact of image distortion and two types of image noise on the navigation performance are investigated.

This paper develops an intelligent flight controller for a fixed-wing aircraft model in the longitudinal plane, using a Reinforcement Learning (RL)-based control method, namely Deep Deterministic Policy Gradient (DDPG). The neural net-work controller is fed the values of aircraft position, velocity, pitch angle and pitch rate, and outputs the elevator deflection. Artificial Neural Network (ANN)s are used to approximate the nonlinear state-action value function and the policy function. Simulation results show that the flight controller learns from the experienced data to fly over an obstacle wall with constrained pitch angle.

Recent research in artificial intelligence potentially provides solutions to the challenging problem of fault-tolerant and robust flight control. This paper proposes a novel Safety-Informed Evolutionary Reinforcement Learning algorithm (SERL), which combines Deep Reinforcement Learning (DRL) and neuroevolution to optimize a population of nonlinear control policies. Using SERL, the work has trained agents to provide attitude tracking on a high-fidelity nonlinear fixed-wing aircraft model. Compared to a state-of-the-art DRL solution, SERL achieves better tracking performance in nine out of ten cases, remaining robust against faults and changes in flight conditions, while providing smoother action signals.

Incremental Nonlinear Dynamic Inversion (INDI) has received substantial interest in the recent years as a nonlinear flight control law design methodology that features inherent robustness against bare airframe aerodynamic variations. However, systematic studies into the robust design benefits of INDI-based control over the classical divide-and-conquer philosophy have been scarce. To bridge this gap, this paper compares the setup of hybrid INDI with a standard industry benchmark that is based on two-degree-of-freedom gain-scheduled proportional-integral-derivative control. This is done on an architectural basis and in terms of achievable robust stability and performance levels with respect to a common set of design requirements. To this end, a non-smooth, multi-objective H_∞-synthesis algorithm is used that incorporates mixed parametric and dynamic uncertainties in the design objective and constraints. It is shown that close similarities exist between hybrid INDI design and gain-scheduled PID control, which leads to virtually equivalent robustness and performance outcomes in both linear time-invariant and linear time-varying contexts. It is therefore concluded that the main benefit of the hybrid INDI does not lie in improved robustness properties per se, but in the opportunity to perform modular robust design in an implicit model-following context. Specifically, this implies that the areas of flying qualities, robustness, and nonlinear implementation are directly visible and accessible in the control law structure.

MegAWES is a reference design and simulation framework for ground-generation, fixed-wing airborne wind energy systems with a nominal power output of 3 MW. The winch size of MegAWES is based on a smaller system and needs to be scaled up because the current size leads to unrealistically fast dynamics, which require saturation. However, there is no available method to select an appropriate size for the winch. Additionally, while it has been hypothesized that the size of the winch has a significant effect on the dynamics of the overall system for ground-generation concepts, this effect has not been quantified. In this work, we first analyze the effects of the winch size on the system dynamics using a linearized model. Second, we present a method to find the upper bound for the size of the winch based on a selected maximum tether force overshoot during nominal operation. Third, we apply this method to find an upper bound for the winch size for the MegAWES reference design. Using the nonlinear MegAWES simulation framework, we validated this upper bound. At the upper bound, the system accurately tracked the reference tether force without overshoot and when exceeding our upper bound, the tether force response was oscillatory and overshot its ideal value.

This paper proposes a novel dynamic programming algorithm for nonlinear system optimal control problem, namely Incremental Generalized Policy Iteration (IGPI). The proposed IGPI algorithm combines the advantages of Incremental Control(IC) and Generalized Policy Iteration(GPI). Incremental control can handle the nonlinearity and uncertainty in nonlinear systems without knowing the nonlinear system information, GPI can learn an optimal control law for dynamical systems. Based on the proposed IGPI algorithm, a data-driven adaptive attitude controller is designed for a spacecraft with sloshing liquid fuel. Simulation results demonstrate the effectiveness of the spacecraft attitude controller.

In this paper, we propose an obstacle avoidance solution for a 34-gram quadcopter equipped with a monocular camera. The perception of obstacles is tackled by a lightweight convolutional neural network predicting a dense depth map from a captured grey-scale image. The depth network performs self-supervised learning and thus requires no ground-truth labels that are costly to acquire. Based on the depth map, the control strategy is implemented by a behavior state machine that balances the efficiency to explore the environment and the safety of avoiding obstacles. In real-world flight experiments, our solution demonstrates the efficacy of predicting trust-worthy depth maps and a stable control strategy in various cluttered environments.

Recent aerospace systems increasingly demand model-free controller synthesis, and autonomous operations require adaptability to uncertainties in partially observable environments. This paper applies distributional reinforcement learning to synthesize risk-sensitive, robust model-free policies for aerospace control. We investigate the use of distributional soft actor-critic (DSAC) agents for flight control and compare their learning characteristics and tracking performance with the soft actor-critic (SAC) algorithm. The results show that (1) the addition of distributional critics significantly improves learning consistency, (2) risk-averse agents increase flight safety by avoiding uncertainties in the environment.

Considerable growth in the number of passengers and cargo transported by air is predicted. Moreover, aircraft noise and climate impact become increasingly important factors in aircraft design. These existing challenges in aviation boost interest in the design of innovative aircraft configurations. One of these configurations is a V-shaped flying wing named the Flying-V. This work aims at developing a flight control system for the Flying-V that can be used to improve the stability and handling qualities of the aircraft. Prior work shows that the Flying-V is not able to adhere to all stability and handling quality requirements at the forward and aft centre of gravity location during cruise and approach. This paper illustrates how an Incremental Nonlinear Dynamic Inversion flight control system can be used to improve the stability and handling qualities of the aircraft. Furthermore, the robustness of the flight control system is assessed by analysing the effects of aerodynamic uncertainty on the attitude tracking error of the Flying-V. Upon implementation of the flight control system, this research shows that the eigenmodes become stable. Besides that, the flight control system is proved to be robust against aerodynamic uncertainty.

Incremental nonlinear dynamic inversion (INDI) is a sensor-based control law design strategy that is based on the principles of feedback linearization. Contrary to its nonincremental counterpart (nonlinear dynamic inversion), this design method does not require a complete onboard model of the airframe dynamics and is therefore more robust against regular perturbations arising from aerodynamic variations. Therefore, INDI brings a natural design approach to desirable flying qualities. However, robustness against singular perturbations, which may arise due to transport delays, elastic airframe effects, or other types of badly modeled or unknown dynamics, is a known challenge for INDI-based control laws. Therefore, this paper addresses the question of robust stability and performance for INDI and its linear form (incremental dynamic inversion [IDI]) in the context of mixed regular and singular perturbations. This is done through analytical insights and by performing quantitative robustness assessments based on the structured singular value framework. Additionally, inversion loop augmentation solutions are investigated using robust synthesis techniques to improve the robustness characteristics of basic IDI designs.

Reinforcement Learning is being increasingly applied to flight control tasks, with the objective of developing truly autonomous flying vehicles able to traverse highly variable environments and adapt to unknown situations or possible failures. However, the development of these increasingly complex models and algorithms further reduces our understanding of their inner workings. This can affect the safety and reliability of the algorithms, as it is difficult or even impossible to determine which are their failure characteristics and how they will react in situations never tested before. It is possible to remedy this lack of understading through the development of eXplainable Artifial Intelligence and eXplainable Reinforcement Learning methods like SHapley Additive Explanations. This tool is used to analyze the strategy learnt by an Actor-Critic Incremental Dual Heuristic Programming controller architecture when presented with a pitch rate or roll rate tracking task in non-linear flying conditions, such as at high angles of attack and large sideslip angles. This same controller architecture has been previously explored with the same analysis tool but limited to the nominal linear flight regime, and it was observed that the controller learnt linear control laws, even though its Artificial Neural Networks should be able to approximate any function. Interestingly, it was discovered in this research paper that even in the non-linear flight regime it is still more optimal for this controller architecture to learn quasi-linear control laws, although it seems to continuously modify the linear slope as if it was an extreme case of the gain scheduling technique.

Enabling the capability of assessing risk and making risk-aware decisions is essential to applying reinforcement learning to safety-critical robots like drones. In this paper, we investigate a specific case where a nano quadcopter robot learns to navigate an apriori-unknown cluttered environment under partial observability. We present a distributional reinforcement learning framework to generate adaptive risk-tendency policies. Specifically, we propose to use lower tail conditional variance of the learnt return distribution as intrinsic uncertainty estimation, and use exponentially weighted average forecasting (EWAF) to adapt the risk-tendency in accordance with the estimated uncertainty. In simulation and real-world empirical results, we show that (1) the most effective risk-tendency varies across states, (2) the agent with adaptive risk-tendency achieves superior performance compared to risk-neutral policy or risk-averse policy baselines. Code and video can be found in this repository: https://github.com/tudelft/risk-sensitive-rl.git