UAVs have become increasingly popular, finding applications in diverse areas such as military operations, search and rescue, delivery services, wireless communication, and aerial surveillance. A critical aspect of UAV operations is autonomous landing, especially on moving targets like ships, which presents significant challenges due to the dynamic and unpredictable maritime environment. This research explores the potential of reinforcement learning as a strategy for achieving optimal guidance during the autonomous landing process of the VSQP. The study illustrates that the reinforcement learning learning framework can effectively steer the VSQP, ensuring safe and accurate landings on a moving ship, and outperforming the benchmark controller in a more intelligent manner. The proposed approach not only improves landing performance but also extends the operational capability of the VSQP.

UAVs have become increasingly popular, finding applications in diverse areas such as military operations, search and rescue, delivery services, wireless communication, and aerial surveillance. A critical aspect of UAV operations is autonomous landing, especially on moving targets like ships, which presents significant challenges due to the dynamic and unpredictable maritime environment. This research explores the potential of reinforcement learning as a strategy for achieving optimal guidance during the autonomous landing process of the VSQP. The study illustrates that the reinforcement learning learning framework can effectively steer the VSQP, ensuring safe and accurate landings on a moving ship, and outperforming the benchmark controller in a more intelligent manner. The proposed approach not only improves landing performance but also extends the operational capability of the VSQP.

Quadrotors have continuously leveraged the use of artificial intelligence for navigation and decision-making. Moreover, neuromorphic computing, specifically Spiking Neural Networks (SNNs), is considered as an energy-efficient solution during inference. The current study will analyse the effects of implementing SNNs for mimicking energy optimal guidance and control. To achieve this, population encoding is used and an equivalent of 7-8 spiking neurons per conventional neuron is found to preserve most of the information. The equivalent controller prefers fast adaptation which requires small spiking threshold values and minimal reliance on past information. To improve the controller performance, dataset selection is of utmost importance with a careful trade-off between excessive race track customisation and generalisability being required. The results show that learning is feasible and SNN performance approaches conventional state-of-the-art models trained with multi-layer perceptrons. The current analysis represent an important step towards the rapid guidance and control of ultra-small energy efficient quadrotors.

Autonomous drone racing has gained attention for its potential to push the boundaries of drone navigation technologies. While much of the existing research focuses on racing in obstacle-free environments, few studies have addressed the complexities of obstacle-aware racing, and approaches presented in these studies often suffer from overfitting, with learned policies generalizing poorly to new environments. This work addresses the challenge of developing a generalizable obstacle-aware drone racing policy using deep reinforcement learning. We propose applying domain randomization on racing tracks and obstacle configurations before every rollout, combined with parallel experience collection in randomized environments to achieve the goal. The proposed randomization strategy is shown to be effective through simulated experiments where drones reach speeds of up to 70 km/h, racing in unseen cluttered environments. This study serves as a stepping stone toward learning robust policies for obstacle-aware drone racing and general-purpose drone navigation in cluttered environments. Code is available at https://github.com/ErcBunny/IsaacGymEnvs.

With their ability to access hard-to-reach spaces and to provide a birds-eye view over large areas,Micro Air Vehicles (MAV) are already taking over awide variety of monitoring and inspection tasks. As the continuing miniaturization of electronic components further drives down cost, recent advancements in the design of control algorithms promise a future, where these small drones will be able to operate fully autonomously and with minimal human input. Spatial awareness is a key aspect in developing fully autonomous MAVs and in this dissertation,we develop new algorithms for on-board localization usingUltra-Wideband (UWB) ranging. UWB is a low-power technology that enables data communication and time-of-flight range measurements. The large bandwidth of UWB results in high timing accuracy as well as an improved resistance to shadowing and multipathing, which makes UWB a very useful technology for ranging in indoor environments...@en

In recent years, the use of drones in practical applications has seen a rapid increase, for instance in inspection, agriculture or environmental research. Most of these drones have a span in the order of tens of centimeters and a weight of half a kilogram or more. Smaller drones offer advantages in terms of safety and cost. However, their reduced payload capacity makes it difficult to carry the sensors and computers required for autonomous operation.

One of the most essential tasks an autonomous drone needs to perform is navigation. Here, navigation is defined as the ability to move towards a specified location while avoiding obstacles along the way. Ideally, the drone should also remember traveled routes, to make the return journey more efficient. However, on tiny drones (palm-size or smaller) the on-board processing power is often limited to a single microcontroller and the choice of sensors is limited. Cameras are popular sensors for tiny drones, because they're small, lightweight and passive, although they do require some processing power to produce useful results. The goal of this dissertation is to find a new, visual navigation strategy that fits within the constraints of these tiny drones.

First, existing work in terms of visual perception and avoidance is reviewed. Multiple options exist for visual perception: stereo vision, optical flow and monocular vision. All of these options are discussed and compared, leading to the conclusion that stereo vision performs best at shorter distances albeit at the cost of an additional camera, while monocular vision performs better at longer distances. Optical flow is ruled out for avoidance, as it has excessively large errors precisely in the direction of movement.
For avoidance, the options in terms of motion planning, map types and odometry are discussed. Perhaps unsurprisingly, the optimal choice is found to be dependent on the application. For computational efficiency on tiny drones, the most important choice is whether multiple measurements should be fused into a single map, or if individual percepts are good enough for avoidance. The latter is significantly less computationally demanding. For visual odometry, the depth information should be used if available, and the IMU can provide efficiency benefits in feature tracking. At Preliminary results are shown for monocular vision, visual odometry and obstacle avoidance.

Secondly, the dissertation takes a deeper dive into monocular depth estimation. Monocular depth estimation has the advantage that it only needs a single camera -- which saves valuable weight on tiny drones -- but its processing is more complex. The goal of this chapter is to analyze the learned behavior of neural networks for monocular depth perception, to see if this can be distilled into simple, lightweight algorithms. Using experiments based on data augmentation, it is shown that all four of the analyzed networks rely on the vertical position of objects in the image to estimate their depth. While this cue would be simple to replicate, it does depend on a known pose of the camera. Further investigation shows that the networks have a strong prior `assumption' about this pose, which may make transfer to drones more difficult. Finally, the networks need to have some sense of an `object'. In this case, it is shown that various shapes are recognized as an object provided that they have contrasting outlines and a dark shadow at the bottom. While this last feature is clearly present in the car-based KITTI dataset, it may not transfer directly to other environments. However, the vertical position cue can likely be used to provide monocular depth estimates to resource-limited systems such as tiny drones.

Thirdly, the remembering of traveled routes is investigated. Traditional mapping strategies from robotics would quickly run out of memory on microcontrollers, especially over longer trajectories. Instead, inspiration for a memory-efficient route-following strategy is found in nature. Here, insects are able to remember and follow remarkably long routes despite their tiny brains. Their strategy is often broken up into a few components, most notably path integration (odometry in robotics) and visual homing. We implement a novel strategy based on these components on a 56-gram drone. Here, the focus lies on traveling long distances using odometry, while periodically using visual homing to return to known locations to counteract odometric drift. The proposed strategy is demonstrated over multiple experiments, where the most efficient run required only 0.65 kilobytes to remember a route of 56 meters. This shows that tiny drones can retrace known paths by combining odometry with periodic homing maneuvers to counteract drift.

Finally, the avoidance of obstacles is discussed in the conclusion of this dissertation. This research has been performed by MSc students under my supervision, who have found and demonstrated that bug algorithms are an effective navigation strategy in three-dimensional, limited-field-of-view applications and provide a lightweight goal-oriented avoidance strategy that is suitable for tiny drones.

By combining all of the above results, a full navigation strategy for tiny drones can be proposed: tiny drones can visually navigate by using lightweight monocular vision algorithms to perceive obstacles, three-dimensional bug algorithms to avoid them while moving to new locations, and odometry and visual homing to retrace known paths.@en

Executing quadrotor trajectories accurately and therefore safely is a challenging task. State-of-the-art adaptive controllers achieve impressive trajectory tracking results with slight performance degradation in varying winds or payloads, but at the cost of computational complexity. Requiring additional embedded computers onboard, adding weight and requiring power. Given the limited computational resources onboard, a trade-off between accuracy and complexity must be considered. To this end, we implement "Neural-Fly" a lightweight adaptive neural controller to adapt to propeller damage, a common occurrence in real-world flight. The adaptive neural architecture consists of two components: (I) offline learning of a condition invariant representation of the aerodynamic forces through Deep Neural Networks (II) fast online adaptation to the current propeller condition using a composite adaptation law. We deploy this flight controller fully onboard the flight controller of the Parrot Bebop 1,showcasing its computational efficiency. The adaptive neural controller improves tracking performance by ≈60% over the nonlinear baseline, with minimal performance degradation of just ≈20% with increasing propeller damage.

Reaching fast and autonomous flight requires computationally efficient and robust algorithms. To this end, we train Guidance & Control Networks to approximate optimal control policies ranging from energy-optimal to time-optimal flight. We show that the policies become more difficult to learn the closer we get to the time-optimal ’bang-bang’ control profile. We also assess the importance of knowing the maximum angular rotor velocity of the quadcopter and show that over- or underestimating this limit leads to less robust flight. We propose an algorithm to identify the current maximum angular rotor velocity onboard and a network that adapts its policy based on the identified limit. Finally, we extend previous work on Guidance & Control Networks by learning to take consecutive waypoints into account. We fly a 4×3m track in similar lap times as the differential-flatness-based minimum snap benchmark controller while benefiting from the flexibility that Guidance & Control Networks offer.

Aerial platforms designed for water jet placement are gaining interest in the areas of fire-fighting, washing, and irrigation. A novel, lightweight, and simplistic design is proposed that reduces the number of actuators and limits ineffective water discharge. External camera feedback was used for position control as a first step towards autonomous flight. An initial prototype of an unmanned hydro-propelled aerial vehicle (UHAV) connected to a water hose was designed and fabricated. Flight tests were conducted to show that attitude control with uniaxial thrust-vectoring of two nozzles was impossible due to undamped vibrations and coupling effects. By redesigning the PID controller, pitch rate damping was accomplished. Furthermore, a design trade-off led to the introduction of a canting keel to reduce bank-yaw coupling effects due to asymmetric nozzle deflections. Flight tests proved that the iterated design with a hose length of 3m was capable of disturbance rejection and setpoint tracking. An external camera was used to show that the Lucas-Kanade optical flow algorithm and the implementation of the YOLOv5 segmentation model can be used for positional water jet placement. By increasing the pitch rate damping, improving the water jet detection algorithm and implementing a cost function for water discharge at the area of interest, autonomous missions can be flown in the future.

In this study, we present a first step towards a cutting-edge software framework that will enable autonomous racing capabilities for nano drones. Through the integration of neural networks tailored for real-time operation on resource-constrained devices. A lightweight Convolutional Neural Network, with the Gatenet architecture, is adjusted for reduced computational demand and is successfully deployed on a GAP8 processor at a rate of 16$Hz$. This network provides gates' size and location data for the subsequent positioning algorithm. A second neural network, trained through reinforcement learning, governs the drone's guidance and control systems, demonstrating a remarkable rate of 167$Hz$ on an STM32F405 processor. The attitude rates and thrust outputted by this network are then fed to an attitude rate PID controller.

The research shows that state-of-the-art neural networks for drone racing can be deployed on nano drones, despite their limited processing power. Nonetheless, the study demonstrated specific limitations, such as the perception network's sensitivity to white pixels in the image reducing its effectiveness when light sources are present in the scene. These findings underscore the importance of dataset composition and the need for diverse training scenarios to enhance the neural network's generalizability and performance in real-world applications.

Developing optimal controllers for aggressive high speed quadcopter flight remains a major challenge in the field of robotics. Recent work has shown that neural networks trained with supervised learning are a good candidate for real-time optimal quadcopter control. In these methods, the networks (termed G\&CNets) are trained using optimal trajectories obtained from a dynamical model of the quadcopter by means of a direct transcription method. A major problem with these methods is the effects of unmodeled dynamics. In this work we identify these effects for G\&CNets trained for power optimal full state-to-rpm feedback. We propose an adaptive control strategy to mitigate the effects of unmodeled roll, pitch and yaw moments. Our method works by generating optimal trajectories with constant external moments added to the model and training a network to learn the policy that maps state and external moments to the corresponding optimal rpm command. We demonstrate the effectiveness of our method by performing power-optimal hover-to-hover flights with and without moment feedback. The flight tests show that the inclusion of this moment feedback significantly improves the controller's performance. Additionally we compare the adaptive controller's performance to a time optimal Bang-Bang controller for consecutive waypoint flight and show significantly faster lap times on a 3x4m track.

Flying-wings show great potential for a vast number of applications, in both commercial and military sectors, thanks to their long range and fast forward flight, but suffer due to their lack of vertical take-off and landing capabilities. This paper presents a proof of concept for a novel landing method for a conventional flying wing that does not introduce additional weight dedicated only to the landing phase, with the aim of controlling a deep-stalled flying-wing in a powered flat spin. Through cyclic actuation of the servo motors and elevons, lateral forces as well as moments can be generated to control the position and attitude of the rotation plane. A successful indoor experiment was performed with a modified Parrot Disco in a controlled environment. Outdoor tests, however, failed to replicate the indoor results due to additional challenges present in the real flight conditions. A number of key challenges were identified, and the insights gained in this research lay an initial foundation for future work on this topic.

Relative localization (RL) is essential for the successful operation of micro air vehicle (MAV) swarms. Achieving accurate 3-D RL in infrastructure-free and GPS-denied environments with only distance information is a challenging problem that has not been satisfactorily solved. In this work, based on the range-based peer-to-peer RL using the ultra-wideband (UWB) ranging technique, we develop a novel UWB-based cooperative relative localization (CRL) solution which integrates the relative motion dynamics of each host-neighbor pair to build a unified dynamic model and takes the distances between the neighbors as bonus information. Observability analysis using differential geometry shows that the proposed CRL scheme can expand the observable subspace compared to other alternatives using only direct distances between the host agent and its neighbors. In addition, we apply the kernel-induced extended Kalman filter (EKF) to the CRL state estimation problem with the novel-designed Logarithmic-Versoria (LV) kernel to tackle heavy-tailed UWB noise. Sufficient conditions for the convergence of the fixed-point iteration involved in the estimation algorithm are also derived. Comparative Monte Carlo simulations demonstrate that the proposed CRL scheme combined with the LV-kernel EKF significantly improves the estimation accuracy owing to its robustness against both the measurement outliers and incorrect measurement covariance matrix initialization. Moreover, with the LV kernel, the estimation is still satisfactory when performing the fixed-point iteration only once for reduced computational complexity.

For most robotics applications, optimal control remains a promising solution for solving complex control tasks. One example is the time-optimal flight of Micro Air Vehicles (MAVs), where strict computational requirements fail to resolve such algorithms onboard. Recent work on the use of deep neural networks for guidance and control (G&CNets) has shown that these biologically inspired models approximate well the optimal control solution while requiring a fraction of the computational cost. Although previous attempts resulted in successful flight tests, training occurred on large-scale datasets based on a 3-DoF model. Since model refinement leads to higher generation time, in this work, we show that G&CNets trained on small-sized datasets can mimic the optimal control solution of a full 6-DoF quadrotor model. The cost function used in the generation process penalizes the altitude error and mixes both time and power-optimal objectives weighted by a varying homotopy parameter. Trained networks output the vertical thrust command and body rates based on the vehicle's position, velocity, and attitude. The proposed controller transfers well onboard for different flight scenarios: (i) longitudinal, lateral and diagonal flight; (ii) hovering with and without the effect of disturbances and (iii) waypoint tracking experiment. Through a Monte-Carlo test campaign, it is demonstrated that G&CNets trained on small datasets provide similar results to those with 100 times more samples. To the best of our knowledge, this work is the first implementation of a high-dimensional G&CNet in the control loop of a real MAV.

As the application areas of Unmanned Aerial Vehicles (UAVs) keep expanding, new flight areas are encountered more often. Small UAVs, named Micro Air Vehicles (MAVs), even fly in areas like sewage pipes. These areas introduce new difficulties such as aerodynamic effects caused by the ground and/or ceiling. In this paper two main contributions are presented that deal with the aerodynamic effects caused by the ceiling: 1) an adaptive model describing the ceiling effect using onboard measurements, which can be altered to describe other aerodynamic effects that occur when flying in constrained spaces, 2) incorporating the adaptive model into an Incremental Nonlinear Dynamic Inversion (INDI) controller. The controller is implemented and tested onto a MAV (Crazyflie). The results have shown stability improvements for close ceiling flight. Moreover the minimal distance the MAV can fly from the ceiling is decreased using the new controller.

The application of Unmanned Aerial Vehicles (UAVs) is increasing, much like the performance of these aircraft. A tailsitter is a type of UAV which is capable of performing vertical take-offs and landings (VTOL) and long endurance flights. During hover, the yaw control is limited due to the dynamics of these tailsitters. The generally used quaternion feedback for the attitude does not compensate for this as it describes a singular rotation. Tilt-twist is a solution to the problem as it splits the tilt (pitch and roll) from the twist (yaw). The axis of the yaw rotation is body fixed. When hovering with a pitch and/or roll angle the twist axis will be aligned with the body z-axis, instead of the desired gravitational force vector (for position control). Previous tilt-twist methods used a PID controller. This paper describes an improvement over previous tilt-twist approaches, the dynamic tilt-twist in combination with INDI. The INDI controller is designed for nonlinear systems. The dynamic tilt-twist compensates for the problem with the normal tilt-twist as test results will demonstrate. Tests are performed in a simulation and a real life test with the NederDrone hybrid tailsitter is done.

This research presents an implementation of a novel controller design on an overactuated hybrid Unmanned Aerial Vehicle (UAV). This platform is a hybrid between a conventional quadcopter and a fixed-wing aircraft. Its inner loop is controlled by an existing but modified control method called Incremental Non-linear Control Allocation or INCA. This controller deals with the platform’s control allocation problem by minimising a set of objective functions with a method known as the Active Set Method and avoids actuator saturation. For the vehicle’s outer loop, a novel extension to INCA is presented, called Extended INCA or XINCA. This method optimises one of the physical actuator’s command and the angular control demands fed to the vehicle’s inner loop, based on linear reference accelerations. It does so while adapting to varying flight phases, conditions and vehicle states, and taking into account the aerodynamic properties of the main wing. XINCA has low dependence on accurate vehicle models and requires configuration using only several optimisation parameters. Both flight simulations and experimental flights are performed to prove the performance of both controllers.

The e-sport of drone-racing involves human pilots to race against time. Recently, drone races have also gone fully-autonomous. As a result, these agile robotic platforms not only pose challenges of flying fast to the participating pilots but also create challenges for the flight control computers. As a result, the concept of autonomous drone racing has gained significant attention from research groups around the world. These races aim to push the boundaries of perception and control algorithms, while simultaneously mitigating the real-world uncertainty of execution on autonomous systems. While perception algorithms face challenges due to limited feature detection, high motion blur and computational requirements, control algorithms face challenges of convergence to the desired trajectories that are planned out in the race arena.
This thesis addresses the challenge of control for racing, which is responsible for guiding the drone to design and track desired trajectories for fast flights. The control sub-modules of racing drones are responsible for generating trajectories for fastest possible flights and also for obeying these generated commands. Additionally, the requirement of limited algorithm complexity is added to match the philosophy of computationally efficient algorithms at the Micro Air Vehicle Laboratory. However, to address the requirements of these control sub-modules, the prerequisite of accurate state estimation always persists. Assigning control actions to a robot without information on the current state of the robot is rather unwise. As a result, this thesis first aims to perform accurate state estimation before designing controllers for time-optimal trajectory tracking. Again, another constraint of using only a single sensor (i.e. the Inertial Measurement Unit) is added to make the drone race in GPS denied environments. As a result, the goal of the thesis is two-fold i.e. making accurate state estimators while using limited sensors and designing optimal controllers for taking the quickest trajectory through the arena. To achieve the goal of accurate state estimation, existing techniques are studied. Several features from each of these methods are selected to design a new estimator. To achieve the goal of time-optimal trajectory generation, firstly, the flaws of traditional control methods are pointed out. A new optimal-control technique is proposed, which makes use of fundamental principles dating back several decades. This principle is then fused along with present-day optimization solvers. Finally, the proposed state estimation and control algorithm are compared against prior (benchmarked) techniques in the area. Compared to existing optimal control techniques, the proposed algorithm leads to faster trajectories and consumes less computational power onboard.

Previous years have seen a rise in the use of Unmanned Aerial Vehicles (UAVs). Reaching a large endurance and range while being able to perform Vertical Take-Off and Landing (VTOL) landings allows a broad range of applications. For this purpose the DelftaCopter (DC) was developed, a tilt-body tailsitter UAV. It hovers using a single helicopter rotor for lift and transitions to forward flight by pitching its body down by 90°. In this forward flight state, wings generate the lift, while the helicopter rotor now provides thrust. The single rotor is more efficient than using multiple smaller rotors and helicopter swashplate is used for attitude and speed control. The heavy single helicopter rotor introduces significant gyroscopic moments, as is the case for all helicopters. In contrast with normal helicopters, the DC has a heavy fuselage putting the attitude dynamics between a helicopter and aircraft. In previous research, a controller based on a model incorporating the rotor as a rotating cylinder was implemented. This controller was unable to counteract the gyroscopic pitch-roll coupling, leading to the question of this thesis: how should the DC be modeled to allow control design.
In this thesis, the previous model is called the Cylinder Dynamics (CD) model, and is compared with another model from literature. The latter model, in this thesis called the Tip-Path Plane (TPP) model, includes the flapping dynamics through the tip-path plane dynamics and is also a linear state-space model. In flight tests, chirps were used to cover a broad frequency range. Fitting both the CD and TPP models on this flight test data, it is shown that the CD model lacks accuracy in the high-frequency area, while the TPP is able to accurately model these dynamics. This shows that the flapping dynamics are important to the attitude dynamics of the DC. An Linear Quadratic Regulator (LQR) controller was implemented based on the fitted TPP model, and shows adequate tracking performance, further validating the applicability of the model to the DC. For forward flight, extensions to the hover models are proposed. The extension including the elevator and aerodynamic damping is shown to simulate key dynamics of the DC in forward flight with reasonable accuracy. The parameters and eigenfrequencies of this model are not significantly different from the hover model. Therefore it can be concluded that the gyroscopic effect plays an important role in forward flight attitude dynamics. Another extension which estimates of angle of attack and sideslip using high-pass filtered rotational rates, yields better accuracy, but significantly changes the model parameters also present in the hover model. More research with angle of attack and sideslip vanes could validate this modeling approach. It was also found that for a new version of the DC with a smaller, more quickly rotating rotor, the modeling done before resulted in much worse fits. It was shown that the CD and TPP model response is much more comparable for this version. Control performance also suffers due to this lower accuracy model fit. Further research is required to understand why this is the case.