In recent years, flapping wing micro aerial vehicles (FWMAVs) have garnered significant attention due to their agility in cluttered environments and the safety advantages offered by their soft wings during close-proximity operations. These vehicles are subject to numerous constraints: limited power supply, restricted payload capacity, high drag and vibration from the flapping mechanism, and susceptibility to environmental disturbances such as wind gusts—making indoor operation preferable. However, these vehicles have not achieved autonomous navigation yet. The platform used in this study, the Flapper Nimble+, is attitude-stable, meaning it lacks access to egomotion or positional information and is only capable of controlling its thrust, pitch, roll, and yaw. Given the strict SWaP (Size, Power and Weight) constraints, lightweight Time-of-Flight (ToF) sensors are among the most practical sensing solutions. This thesis investigates the question: How can obstacle avoidance be achieved in an attitude-stable flapping wing air vehicle equipped with Time-of-Flight sensors? Two control approaches were implemented and tested in the IsaacGym simulator: a simple PID controller with confidence-based yaw adjustment, and a reinforcement learning (RL) policy trained using Proximal Policy Optimization. Two sensor configurations were considered: a minimal two-sensor setup (front and downward) and a richer five-sensor array for broader perception. The PID controller successfully avoided collisions in all environments, demonstrating high reliability but limited exploration, averaging 35% coverage. In contrast, the RL policy with five sensors achieved greater spatial exploration—averaging 49.5% coverage—at the cost of an average of 3.5 collisions per episode. When reduced to two sensors, the RL agent’s performance declined significantly, with average coverage dropping to 29% and collision counts increasing to 6.6 (excluding failed runs). These results show that autonomous navigation is achievable for attitude-stable flapping wing air vehicles using only ToF sensors. While PID control offers reliable, conservative navigation under minimal sensing, RL enables more exploratory behaviors and adaptability in dynamic environments.

he real-world application of Micro Air Vehicles (MAVs) is often constrained by their limited flight range and endurance, primarily due to battery limitations. One way to overcome this challenge is by leveraging naturally occurring vertical air currents, a technique inspired by soaring birds. This paper introduces Osprey Simulator, a framework designed to simulate and test energy-efficient soaring flight strategies for fixed-wing drones. Built on Isaac Sim, Osprey Simulator enables the creation of both randomly generated urban environments and real-world environments, such as the TU Delft campus. These environments are paired with accurate wind field simulations using OpenFOAM, providing a robust platform for studying aerodynamic interactions in diverse settings. This functionality allows for the generation of scalable synthetic datasets, including depth images and wind field information, enabling comprehensive exploration of soaring potential across various structural geometries and wind conditions. Using generated synthetic data, a neural network was trained to predict optimal soaring regions by analyzing depth images and wind field information. The network demonstrates the ability to identify updraft and downdraft regions, enabling more efficient path planning for drones in urban environments. By integrating realistic simulations and advanced predictive models, Osprey Simulator serves as a powerful tool for advancing autonomous soaring and extending the operational range of fixed-wing MAVs.

To safely and efficiently solve motion planning problems in multi-agent settings, most approaches attempt to solve a joint optimization that explicitly accounts for the responses triggered in other
agents. This often results in solutions with an exponential computational complexity, making these methods intractable for complex scenarios with many agents. While sequential predict-and-plan approaches are more scalable, they tend to perform poorly in highly interactive environments. This paper proposes a method to improve the interactive capabilities of sequential predict-and-plan methods in multi-agent navigation problems by introducing predictability as an optimization objective. We interpret predictability through the use of general prediction models, by allowing agents to predict themselves and estimate how they align with these external predictions. We formally introduce this behavior through the free-energy of the system, which reduces (under appropriate bounds) to the Kullback-Leibler divergence between plan and prediction, and use this as a penalty for unpredictable trajectories. The proposed interpretation of predictability allows agents to more robustly leverage prediction models, and fosters a ‘soft social convention' that accelerates agreement on coordination strategies without the need of explicit high level control or communication. We show how this predictability-aware planning leads to lower-cost trajectories and reduces planning effort in a set of multi-robot problems, including autonomous driving experiments with human driver data, where we show that the benefits of considering predictability apply even when only the ego-agent uses this strategy.

Insects like honeybees exhibit remarkable navigational abilities despite their simple nervous systems, showcasing expertise in tasks such as long-distance travel, landmark recognition, and spatial memory. These skills are crucial for efficient foraging and homing. In robotics, one of the main challenges is to navigate in GPS-denied environments with limited sensors and processors onboard. In this study, we propose a novel navigation strategy that uses learning flights during which a robot directly maps images to nest location vectors, inspired by honeybees. In our previous research, the learning phase was modeled using compact convolutional neural networks (CNNs) and demonstrated successful learning and control in simulation. In this work, we combine this homing model with odometry and implement both in a real-world quadrotor. More specifically, we examine how this model can compensate for the drone’s odometric drift to reach home after it performs a long-distance outbound flight. Real-world experiments demonstrate the proof of concept of the proposed navigation strategy in both indoor and outdoor flights.

This paper presents an encoder-decoder-style convolutional neural network (CNN) for the purpose of improving monocular and stereo depth estimation (SDE) estimates, by combining them with the corresponding monocular estimates through a fusion network, assisted by prior information to provide context for the fusion. Video cameras are commonly used for depth perception in robotics, especially weight-sensitive applications, such as on Micro Aerial Vehicles (MAV). The two primary paradigms for vision-based depth perception are monocular and stereo depth or disparity estimation, each having their own strengths and weaknesses. These strengths and weaknesses seem to be complementary, and thus a fusion of the two may result in more accurate predictions. In this paper, we investigate this fusion by training a CNN that combines stereo and monocular depth or disparity estimates. The fusion network is agnostic to the choice of the input networks, providing great flexibility. It was found that such a fusion network, while increasing the computational complexity of the depth perception pipeline, indeed improves the accuracy of the estimates. The number of outlier predictions has been significantly decreased, while also limiting some fundamental limitations of both stereo and monocular methods, such as errors arising from occluded regions.

Spiking neural networks implemented for sensing and control of robots have the potential to achieve lower latency and power consumption by processing information sparsely and asynchronously. They have been used on neuromorphic devices to estimate optical flow for micro air vehicles navigation, however robotic implementations have been limited to hardware setups with sensing and processing as separate systems. This article investigates a new approach for training a spiking neural network for optical flow to be deployed on the speck2e device from Synsense. The method takes into account the restrictions of the speck2e in terms of network architecture, neuron model, and number of synaptic operations and it involves training a recurrent neural network with ReLU activation functions, which is subsequently converted into a spiking network. A system of weight rescaling is applied after conversion, to ensure optimal information flow between the layers. Our study shows that it is possible to estimate optical flow with Integrate-and-Fire neurons. However, currently, the optical flow estimation performance is still hampered by the number of synaptic operations. As a result, the network presented in this work is able to estimate optical flow in a range of [-4, 1] pixel/s.

New insights into the landing behavior of bumblebees show an adaptive strategy where the optical flow expansion of the landing target is step-wise regulated. In this article, the potential benefits of this approach are studied by replicating the landing experiment with a quadrotor. To this end, an open-loop switching method is developed, enabling fast steps in divergence. An adaptive control law is used to deal with non-linear system dynamics, where the control gain is scheduled based on the control effectiveness of the actuator inputs during the steps. It is demonstrated that the quadrotor can reliably land on the target from varying initial positions, and the switching strategy shows a slight reduction in landing time compared to a constant divergence strategy with the same average divergence over distance. This strategy also reduces the maximum velocity during the landing.

Enabling embodied intelligence in robotics presents several unique challenges. A first major concern is the need for energy efficiency, low latency, and strong temporal reasoning to facilitate effective real-world interaction. Neuromorphic computing has garnered attention as a potential solution to these problems. Secondly, when using deep neural networks, it is hard to shape a learning signal, due to the goal oriented nature of robotics. Reinforcement learning (RL) poses itself as a framework to leverage goal-directed reward functions to create this learning signal.
A key challenge with recurrent and spiking neural networks trained via RL is achieving stable baseline performance, able to creating sequences long enough to stabilize hidden states. This stabilization is crucial for processing sequences that extend beyond the initial warm-up period of the temporal network. In this article, an online RL approach is proposed, enabling temporal training with minimal changes to existing online algorithms, introducing a secondary guiding policy whose sole objective is to prevent episode termination before the warm-up period is complete. This framework is demonstrated to outperform offline RL methods and significantly improve the wall clock time of online RL methods, adapted to sample sequences rather than single transitions. Next, the effect of surrogate gradients as a technique for translating the learning signal from the RL framework to weight updates is analyzed. It is found that the slope, parametrizing the surrogate gradient, plays a crucial role in online RL settings, and can be exploited as an exploration mechanism.

We conduct a simulation study of an insect-inspired navigation method that combines visual learning in a small area around a home location with path integration to successfully navigate over distances 8 to 10 times larger than the learning radius, while only requiring 6MB of memory. Our simulations show that the method is capable of learning with on-board data only and handling the noise levels and errors it may encounter in real-world conditions. Furthermore, we investigate the integration of online learning with a structured buffer to mitigate catastrophic forgetting, allowing continuous learning during flight. Additional experiments assess the system's generalization capabilities beyond the learned area, its robustness to small variations in pitch and roll, and the relationship between network size and the learnable area size.

Insect pest elimination through MAV interception can reduce the need for insecticides and can contribute to sustainable agriculture. In this research, we analyze the feasibility of such solutions through simulated two-player differential games of pursuit and evasion with agents operating on minimalistic sets of biologically-plausible observations and optimized to control constrained vehicle models through deep multi-agent reinforcement learning. Our pursuer and evader agent, representing the quadcopter drone and insect pest respectively, are asymmetric in design, capabilities and objectives. Our results show that our quadcopter pursuer is consistently able to pursue and intercept a reactive insect-inspired evader as well as recordings of actual insect targets, achieving interception rates of 55% and 94% on these respective tasks. In comparison, pursuers alternatively optimized against non-reactive evaders or reactive drone-like evaders with symmetric capabilities, achieve an interception rate of only 42% for the same insect target recordings. Despite these promising results, we conclude that further research is needed to formally establish the superiority of multi-agent optimization in this asymmetric game scenario. Finally, we determine how emergent behavior and strategies resemble nature. During the confrontations, we observe that our pursuer mainly implements pure-pursuit as well as motion camouflage to some degree; drawing comparison to the hunting strategy of dragonfly.

With their ability to access hard-to-reach spaces and to provide a birds-eye view over large areas,Micro Air Vehicles (MAV) are already taking over awide variety of monitoring and inspection tasks. As the continuing miniaturization of electronic components further drives down cost, recent advancements in the design of control algorithms promise a future, where these small drones will be able to operate fully autonomously and with minimal human input. Spatial awareness is a key aspect in developing fully autonomous MAVs and in this dissertation,we develop new algorithms for on-board localization usingUltra-Wideband (UWB) ranging. UWB is a low-power technology that enables data communication and time-of-flight range measurements. The large bandwidth of UWB results in high timing accuracy as well as an improved resistance to shadowing and multipathing, which makes UWB a very useful technology for ranging in indoor environments...@en

In recent years, the use of drones in practical applications has seen a rapid increase, for instance in inspection, agriculture or environmental research. Most of these drones have a span in the order of tens of centimeters and a weight of half a kilogram or more. Smaller drones offer advantages in terms of safety and cost. However, their reduced payload capacity makes it difficult to carry the sensors and computers required for autonomous operation.

One of the most essential tasks an autonomous drone needs to perform is navigation. Here, navigation is defined as the ability to move towards a specified location while avoiding obstacles along the way. Ideally, the drone should also remember traveled routes, to make the return journey more efficient. However, on tiny drones (palm-size or smaller) the on-board processing power is often limited to a single microcontroller and the choice of sensors is limited. Cameras are popular sensors for tiny drones, because they're small, lightweight and passive, although they do require some processing power to produce useful results. The goal of this dissertation is to find a new, visual navigation strategy that fits within the constraints of these tiny drones.

First, existing work in terms of visual perception and avoidance is reviewed. Multiple options exist for visual perception: stereo vision, optical flow and monocular vision. All of these options are discussed and compared, leading to the conclusion that stereo vision performs best at shorter distances albeit at the cost of an additional camera, while monocular vision performs better at longer distances. Optical flow is ruled out for avoidance, as it has excessively large errors precisely in the direction of movement.
For avoidance, the options in terms of motion planning, map types and odometry are discussed. Perhaps unsurprisingly, the optimal choice is found to be dependent on the application. For computational efficiency on tiny drones, the most important choice is whether multiple measurements should be fused into a single map, or if individual percepts are good enough for avoidance. The latter is significantly less computationally demanding. For visual odometry, the depth information should be used if available, and the IMU can provide efficiency benefits in feature tracking. At Preliminary results are shown for monocular vision, visual odometry and obstacle avoidance.

Secondly, the dissertation takes a deeper dive into monocular depth estimation. Monocular depth estimation has the advantage that it only needs a single camera -- which saves valuable weight on tiny drones -- but its processing is more complex. The goal of this chapter is to analyze the learned behavior of neural networks for monocular depth perception, to see if this can be distilled into simple, lightweight algorithms. Using experiments based on data augmentation, it is shown that all four of the analyzed networks rely on the vertical position of objects in the image to estimate their depth. While this cue would be simple to replicate, it does depend on a known pose of the camera. Further investigation shows that the networks have a strong prior `assumption' about this pose, which may make transfer to drones more difficult. Finally, the networks need to have some sense of an `object'. In this case, it is shown that various shapes are recognized as an object provided that they have contrasting outlines and a dark shadow at the bottom. While this last feature is clearly present in the car-based KITTI dataset, it may not transfer directly to other environments. However, the vertical position cue can likely be used to provide monocular depth estimates to resource-limited systems such as tiny drones.

Thirdly, the remembering of traveled routes is investigated. Traditional mapping strategies from robotics would quickly run out of memory on microcontrollers, especially over longer trajectories. Instead, inspiration for a memory-efficient route-following strategy is found in nature. Here, insects are able to remember and follow remarkably long routes despite their tiny brains. Their strategy is often broken up into a few components, most notably path integration (odometry in robotics) and visual homing. We implement a novel strategy based on these components on a 56-gram drone. Here, the focus lies on traveling long distances using odometry, while periodically using visual homing to return to known locations to counteract odometric drift. The proposed strategy is demonstrated over multiple experiments, where the most efficient run required only 0.65 kilobytes to remember a route of 56 meters. This shows that tiny drones can retrace known paths by combining odometry with periodic homing maneuvers to counteract drift.

Finally, the avoidance of obstacles is discussed in the conclusion of this dissertation. This research has been performed by MSc students under my supervision, who have found and demonstrated that bug algorithms are an effective navigation strategy in three-dimensional, limited-field-of-view applications and provide a lightweight goal-oriented avoidance strategy that is suitable for tiny drones.

By combining all of the above results, a full navigation strategy for tiny drones can be proposed: tiny drones can visually navigate by using lightweight monocular vision algorithms to perceive obstacles, three-dimensional bug algorithms to avoid them while moving to new locations, and odometry and visual homing to retrace known paths.@en

Insects have long been recognized for their ability to navigate and return home using visual cues from their nest’s environment. However, the precise mechanism underlying this remarkable homing skill remains a subject of ongoing investigation. Drawing inspiration from the learning flights of honey bees and wasps, we propose a robot navigation method based on directly learning the home vector directions from visual percepts during the learning flight. Subsequently, the robot will travel away from the nest, come back by odometric means, and eliminate the resultant drift by inferring the home vector orientation from the currently experienced view. In this study, a convolutional neural network is employed as learning mechanism in both simulated and real forest environments. Additionally, a comprehensive performance analysis reveals that the network’s homing abilities closely resemble those observed in real insects, all while only utilizing visual and odometric senses. If all images contain sufficient texture and illumination, the average errors
of the inferred home vectors remain below 24°. Moreover, our investigation reveals a noteworthy insight: the trajectory followed during the initial learning flight, for sample image acquisition, exerts a pronounced impact on the network’s output. For instance, a higher density of sample points in proximity to the nest results in a more consistent return.

Event cameras are novel bio-inspired sensors that capture motion dynamics with much higher temporal resolution than traditional cameras, since pixels react asynchronously to brightness changes. They are therefore better suited for tasks involving motion such as motion segmentation. However, training event-based networks still represents a difficult challenge, as obtaining ground truth is very expensive and error-prone. In this article, we introduce EV-LayerSegNet, the first self-supervised CNN for event-based motion segmentation. Inspired by a layered representation of the scene dynamics, we show that it is possible to learn affine optical flow and segmentation masks separately, and use them to deblur the input events. The deblurring quality is then measured and used as self-supervised learning loss.

Although deep reinforcement learning (DRL) is a highly promising approach to learning robotic vision-based control, it is plagued by long training times. This report introduces a DRL setup that relies on self-supervised learning for extracting depth information valuable for navigation. Specifically, a literature study is conducted to investigate the effects of learning how to synthesize one view from the other in a stereo-vision setup without relying on any preliminary knowledge of the camera extrinisics and how it can be integrated for its downstream use for an obstacle avoidance task. As such, the literature study concludes that competitive geometry-free monocular-to-stereo image view synthesis is feasible due to recent developments in computer vision. The scientific paper further develops concepts proposed in the literature study and benchmarks the proposed architectures on depth estimation benchmarks for KITTI. Competitive results are achieved for view synthesis and despite sub-optimal performance compared to state-of-the-art monocular depth estimation, an ability to encode depth and detect shapes is present and, therefore, satisfactory for the application to DRL. Additionally, the research examines the benefits of using the latent space of a view synthesis architecture compared to other feature extractor methods as an input to the PPO agent implemented as auxiliary tasks. This method achieves quicker convergence and better performance for an obstacle avoidance task in a simulated indoor environment than the autoencoding feature extractor and end-to-end DRL methods. It is only outperformed by the monocular depth estimation feature extractor method. Overall, this research provides valuable insights for developing more efficient and effective DRL methods for monocular camera-based drones. Finally, the complementary code for this research can be found: \url{https://github.com/ldenridder/drl-obstacle-avoidance-view-synthesis}.

Prolonging the endurance of fixed-wing UAVs is crucial for achieving complex missions, yet their limited battery life poses a significant challenge. In response, this research proposes a novel approach to extend the endurance of fixed-wing UAVs by enabling autonomous soaring in an orographic wind field. The goal of our research is to develop a controller that can identify feasible soaring regions and autonomously maintain position control without using any throttle. Soaring flight is desirable as it results in a low energy cost with zero throttle usage. However, without throttle usage, the longitudinal motion of the UAV is an under-actuated system, presenting control challenges. The concept of a target gradient line (TGL) is introduced as part of the control algorithm that addresses these challenges and autonomously finds the equilibrium soaring position where sink rate and updraft are in equilibrium. Experimental tests showed promising results, demonstrating the controller’s effectiveness in maintaining autonomous soaring flight in a non-static wind field. We also demonstrate a single degree of control freedom in the soaring position through manipulation of the TGL.

Reaching fast and autonomous flight requires computationally efficient and robust algorithms. To this end, we train Guidance & Control Networks to approximate optimal control policies ranging from energy-optimal to time-optimal flight. We show that the policies become more difficult to learn the closer we get to the time-optimal ’bang-bang’ control profile. We also assess the importance of knowing the maximum angular rotor velocity of the quadcopter and show that over- or underestimating this limit leads to less robust flight. We propose an algorithm to identify the current maximum angular rotor velocity onboard and a network that adapts its policy based on the identified limit. Finally, we extend previous work on Guidance & Control Networks by learning to take consecutive waypoints into account. We fly a 4×3m track in similar lap times as the differential-flatness-based minimum snap benchmark controller while benefiting from the flexibility that Guidance & Control Networks offer.

Decentralised drone swarms need real time collision avoidance, thus requiring efficient, real time relative localisation. This paper explores different data inputs for vision based relative localisation. It introduces a novel dataset generated in Blender, providing ground truth optic flow and depth. Comparisons to MPI Sintel, an industry/research standard optic flow dataset, show it to be a challenging and realistic dataset. Two Deep Neural Network (DNN) architectures (YOLOv3 & U-Net) were trained on this data, comparing optic flow to colour images for relative positioning. The results indicate that using optic flow provides a significant advantage in relative localisation. The flow based YOLOv3 had an mAP of 48%, 9% better than the RGB based YOLOv3, and 23% better than its equivalent U-Net. Its IoU_0.5 of 63% was also 14% better than the RGB based YOLOv3, and 51% than its equivalent U-Net. As an input, it generalises better than RGB, as test clips with variant drones show. For these variants, the optical flow based networks outperformed the RGB based networks by a factor of 10.

In this study, we present a first step towards a cutting-edge software framework that will enable autonomous racing capabilities for nano drones. Through the integration of neural networks tailored for real-time operation on resource-constrained devices. A lightweight Convolutional Neural Network, with the Gatenet architecture, is adjusted for reduced computational demand and is successfully deployed on a GAP8 processor at a rate of 16$Hz$. This network provides gates' size and location data for the subsequent positioning algorithm. A second neural network, trained through reinforcement learning, governs the drone's guidance and control systems, demonstrating a remarkable rate of 167$Hz$ on an STM32F405 processor. The attitude rates and thrust outputted by this network are then fed to an attitude rate PID controller.

The research shows that state-of-the-art neural networks for drone racing can be deployed on nano drones, despite their limited processing power. Nonetheless, the study demonstrated specific limitations, such as the perception network's sensitivity to white pixels in the image reducing its effectiveness when light sources are present in the scene. These findings underscore the importance of dataset composition and the need for diverse training scenarios to enhance the neural network's generalizability and performance in real-world applications.