To safely and efficiently solve motion planning problems in multi-agent settings, most approaches attempt to solve a joint optimization that explicitly accounts for the responses triggered in other
agents. This often results in solutions with an exponential computational complexity, making these methods intractable for complex scenarios with many agents. While sequential predict-and-plan approaches are more scalable, they tend to perform poorly in highly interactive environments. This paper proposes a method to improve the interactive capabilities of sequential predict-and-plan methods in multi-agent navigation problems by introducing predictability as an optimization objective. We interpret predictability through the use of general prediction models, by allowing agents to predict themselves and estimate how they align with these external predictions. We formally introduce this behavior through the free-energy of the system, which reduces (under appropriate bounds) to the Kullback-Leibler divergence between plan and prediction, and use this as a penalty for unpredictable trajectories. The proposed interpretation of predictability allows agents to more robustly leverage prediction models, and fosters a ‘soft social convention' that accelerates agreement on coordination strategies without the need of explicit high level control or communication. We show how this predictability-aware planning leads to lower-cost trajectories and reduces planning effort in a set of multi-robot problems, including autonomous driving experiments with human driver data, where we show that the benefits of considering predictability apply even when only the ego-agent uses this strategy.

This paper presents an encoder-decoder-style convolutional neural network (CNN) for the purpose of improving monocular and stereo depth estimation (SDE) estimates, by combining them with the corresponding monocular estimates through a fusion network, assisted by prior information to provide context for the fusion. Video cameras are commonly used for depth perception in robotics, especially weight-sensitive applications, such as on Micro Aerial Vehicles (MAV). The two primary paradigms for vision-based depth perception are monocular and stereo depth or disparity estimation, each having their own strengths and weaknesses. These strengths and weaknesses seem to be complementary, and thus a fusion of the two may result in more accurate predictions. In this paper, we investigate this fusion by training a CNN that combines stereo and monocular depth or disparity estimates. The fusion network is agnostic to the choice of the input networks, providing great flexibility. It was found that such a fusion network, while increasing the computational complexity of the depth perception pipeline, indeed improves the accuracy of the estimates. The number of outlier predictions has been significantly decreased, while also limiting some fundamental limitations of both stereo and monocular methods, such as errors arising from occluded regions.

Enabling embodied intelligence in robotics presents several unique challenges. A first major concern is the need for energy efficiency, low latency, and strong temporal reasoning to facilitate effective real-world interaction. Neuromorphic computing has garnered attention as a potential solution to these problems. Secondly, when using deep neural networks, it is hard to shape a learning signal, due to the goal oriented nature of robotics. Reinforcement learning (RL) poses itself as a framework to leverage goal-directed reward functions to create this learning signal.
A key challenge with recurrent and spiking neural networks trained via RL is achieving stable baseline performance, able to creating sequences long enough to stabilize hidden states. This stabilization is crucial for processing sequences that extend beyond the initial warm-up period of the temporal network. In this article, an online RL approach is proposed, enabling temporal training with minimal changes to existing online algorithms, introducing a secondary guiding policy whose sole objective is to prevent episode termination before the warm-up period is complete. This framework is demonstrated to outperform offline RL methods and significantly improve the wall clock time of online RL methods, adapted to sample sequences rather than single transitions. Next, the effect of surrogate gradients as a technique for translating the learning signal from the RL framework to weight updates is analyzed. It is found that the slope, parametrizing the surrogate gradient, plays a crucial role in online RL settings, and can be exploited as an exploration mechanism.

Insect pest elimination through MAV interception can reduce the need for insecticides and can contribute to sustainable agriculture. In this research, we analyze the feasibility of such solutions through simulated two-player differential games of pursuit and evasion with agents operating on minimalistic sets of biologically-plausible observations and optimized to control constrained vehicle models through deep multi-agent reinforcement learning. Our pursuer and evader agent, representing the quadcopter drone and insect pest respectively, are asymmetric in design, capabilities and objectives. Our results show that our quadcopter pursuer is consistently able to pursue and intercept a reactive insect-inspired evader as well as recordings of actual insect targets, achieving interception rates of 55% and 94% on these respective tasks. In comparison, pursuers alternatively optimized against non-reactive evaders or reactive drone-like evaders with symmetric capabilities, achieve an interception rate of only 42% for the same insect target recordings. Despite these promising results, we conclude that further research is needed to formally establish the superiority of multi-agent optimization in this asymmetric game scenario. Finally, we determine how emergent behavior and strategies resemble nature. During the confrontations, we observe that our pursuer mainly implements pure-pursuit as well as motion camouflage to some degree; drawing comparison to the hunting strategy of dragonfly.

Decentralised drone swarms need real time collision avoidance, thus requiring efficient, real time relative localisation. This paper explores different data inputs for vision based relative localisation. It introduces a novel dataset generated in Blender, providing ground truth optic flow and depth. Comparisons to MPI Sintel, an industry/research standard optic flow dataset, show it to be a challenging and realistic dataset. Two Deep Neural Network (DNN) architectures (YOLOv3 & U-Net) were trained on this data, comparing optic flow to colour images for relative positioning. The results indicate that using optic flow provides a significant advantage in relative localisation. The flow based YOLOv3 had an mAP of 48%, 9% better than the RGB based YOLOv3, and 23% better than its equivalent U-Net. Its IoU_0.5 of 63% was also 14% better than the RGB based YOLOv3, and 51% than its equivalent U-Net. As an input, it generalises better than RGB, as test clips with variant drones show. For these variants, the optical flow based networks outperformed the RGB based networks by a factor of 10.

In recent years, Artificial Neural Networks (ANN) have become a standard in robotic control. However, a significant drawback of large-scale ANNs is their increased power consumption. This becomes a critical concern when designing autonomous aerial vehicles, given the stringent constraints on power and weight. Especially in the case of blimps, known for their extended endurance, power-efficient control methods are essential. Spiking neural networks (SNN) can provide a solution, facilitating energy-efficient and asynchronous event-driven processing.
In this paper, we have evolved SNNs for accurate altitude control of a non-neutrally buoyant indoor blimp, relying solely on onboard sensing and processing power. The blimp's altitude tracking performance significantly improved compared to prior research, showing reduced oscillations and a minimal steady-state error. The parameters of the SNNs were optimized via an evolutionary algorithm, using a Proportional-Derivative-Integral (PID) controller as the target signal. We developed two complementary SNN controllers while examining various hidden layer structures. The first controller responds swiftly to control errors, mitigating overshooting and oscillations, while the second minimizes steady-state errors due to non-neutral buoyancy-induced drift. Despite the blimp's drivetrain limitations, our SNN controllers ensured stable altitude control, employing only 160 spiking neurons.

We present a computationally cheap 3D bug algorithm for drones, using stereo vision. Obstacle avoidance is important, but difficult for robots with limited resources, such as drones. Stereo vision requires less weight and power than active distance measurement sensors, but typically has a limited Field of View (FoV). In addition, the stereo camera is fixed on the drone, preventing sensor movement. For obstacle avoidance, bug algorithms require few resources. We base our proposed algorithm, Frustumbug, on the Wedgebug algorithm, since this bug algorithm copes with a limited FoV. Since Wedgebug only focuses on 2D problems, the Local-epsilon-Tangent-Graph (LETG) is used to extend the path planning to 3D. Disparity images are obtained through an optimised stereo block matching algorithm. Obstacles are expanded in disparity space to obtain the configuration space. Furthermore, Frustumbug has an improved robustness to noisy range sensor data, and includes reversing, climbing and descending manoeuvres to avoid or escape local minima. The algorithm has been extensively tested with 225 flights in two challenging simulated environments, with a success rate of 96%. Here, 3.6% did not reach the goal and 0.4% collided. Frustumbug has been implemented on a 20 gram stereo vision system, and guides drones safely around obstacles in the real world, showing its potential for small drones to reach their targets fully autonomously.

Semantic segmentation methods have been developed and applied to single images for object segmentation. However, for robotic applications such as high-speed agile Micro Air Vehicles (MAVs) in Autonomous Drone Racing (ADR), it is more interesting to consider temporal information as video sequences are correlated over time. In this work, we evaluate the performance of state-of-the-art methods such as Recurrent Neural Networks (RNNs), 3D Convolutional Neural Networks (CNNs), and optical flow for video semantic segmentation in terms of accuracy and inference speed on three datasets with different camera motion configurations. The results show that using an RNN with convolutional operators outperforms all methods and achieves a performance boost of 10.8% on the KITTI (MOTS) dataset with 3 degrees of freedom (DoF) motion and a small 0.6% improvement on the CyberZoo dataset with 6 DoF motion over the single-frame-based semantic segmentation method. The inference speed was measured on the CyberZoo dataset, achieving 321 fps on an NVIDIA GeForce RTX 2060 GPU and 30 fps on an NVIDIA Jetson TX2 mobile computer.

Widespread usage of Micro Aerial Vehicles (MAVs) has led to various airspace safety breaches, including near mid-air collisions with other aircraft. To ensure safe integration into general aviation, it is paramount that MAVs are equipped with an autonomous detect and avoid system when flying beyond the visual line-of-sight of the operator. The purpose of this research is to investigate the feasibility of acoustic-based aircraft detection, which has generally been overlooked in favor of optical or radar-based technology. Effective sound-based aircraft detection on-board an MAV requires suppressing the dynamic ego-noise it generates during flight, which would otherwise pollute the recorded environmental sound. This paper proposes using a recurrent neural network to predict the generated noise, given a sequence of MAV flight data, so that it can be effectively removed from noisy recordings. For aircraft detection, a convolutional neural network in combination with Mel spectrogram features is designed to classify noise-free environmental sound as either aircraft or non-aircraft, achieving 97.5\% accuracy. To reconstruct the noisy environment of an MAV flight, these noise-free sounds are mixed with ego-noise at mix ratios up to 1.00. When evaluating in these mismatched conditions, accuracy decreases to 95.0\% and 47.5\% with- and without ego-noise suppression, respectively. Although ego-noise suppression can not prevent a drop in performance, the large difference between the mismatched conditions does demonstrate the benefits of the proposed denoising approach on aircraft detection.

Time-optimal model-predictive control is essential in achieving fast and adaptive quadcopter flight. Due to the limited computational performance of onboard hardware, aggressive flight approaches have relied on off-line trajectory optimization processes or non time-optimal methods. In this work we propose a computational efficient model predictive controller (MPC) that approaches time-optimal flight and runs onboard a consumer quadcopter. The proposed controller is built on the principle that constrained optimal control problems (OCPs) have a so-called 'bang-bang' solution. Our solution plans a bang-bang maneuver in the critical direction while aiming for a 'minimum-effort' approach in non-critical direction. Control parameters are computed by means of a bisection scheme using an analytical path prediction model. The controller has been compared with a classical PID controller and theoretical time-optimal trajectories in simulations. We identify the consequences of the OCP simplifications and propose a method to mitigate one of these effects. Finally, we have implemented the proposed controller onboard a consumer quadcopter and performed indoor flights to compare the controller's performance to a PID controller. Flight experiments have shown that the controller runs at 512hz onboard a Parrot Bebop quadcopter and is capable of fast, saturated flight, outperforming traditional PID controllers in waypoint-to-waypoint flight while requiring only minimal knowledge of the quadcopter's dynamics.

Nano quadcopters are ideal for gas source localization (GSL) as they are cheap, safe and agile. However, previous algorithms are unsuitable for nano quadcopters, as they rely on heavy sensors, require too large computational resources, or only solve simple scenarios without obstacles. In this work, we propose a novel bug algorithm named `Sniffy Bug', that allows a swarm of gas-seeking nano quadcopters to localize a gas source in an unknown, cluttered and GPS-denied environment. Sniffy Bug is capable of efficient GSL with extremely little sensory input and computational resources, operating within the strict resource constraints of a nano quadcopter. The algorithm foresees in the avoidance of obstacles and other swarm members, while pursuing desired waypoints. The waypoints are set for exploration, and, when a single swarm member has sensed the gas, by a particle swarm optimization-based procedure. We evolve all the parameters of the bug (and PSO) algorithm, with a novel automated end-to-end simulation and benchmark platform, AutoGDM. This platform enables fully automated end-to-end environment generation and gas dispersion modelling (GDM), not only allowing for learning in simulation but also providing the first GSL benchmark. We show that evolved Sniffy Bug outperforms manually selected parameters in challenging, cluttered environments in the real world. To this end, we show that a lightweight and mapless bug algorithm can be evolved to complete a complex task, and enable the first fully autonomous swarm of collaborative gas-seeking nano quadcopters.

This study investigates the wing deformation of the flapping-wing micro air vehicle (MAV) DelFly II in various flight configurations. Experiments were carried out with the MAV tethered in a windtunnel test section. To determine the best suited measurement approach, a trade-off study was carried out which showed that a point tracking approach with background illumination is most suitable. The therefore used high-speed camera pair and illumination were mounted on the same rotating frame with the DelFly, which allowed adequate viewing axes of the wings at for all pitch angles. Processing was done a purpose-build algorithm, allowing 136 points per wing to be measured simultaneously with an average lost point ratio of 3.4 % and an estimated accuracy of 0.25 mm. Results of hovering flight show some previously unnoticed behaviors. First, it was noted that the upper and lower wing on each side do not deform purely symmetric but show some considerable asymmetric behavior like heave and camber production. Furthermore, the upper wing shows a torsional wave and recoil behavior at faster flapping frequencies, which was shown to be beneficial in insect flight. Lastly, it was found that an air-buffer remains present between the wing surfaces at all times of the clap-and-peel motion (apart from the root trailing edge). This air-buffer increases once freestream velocity is added, which is investigated during the climbing flight study. Here, the reduced angle of attack of the wing is assumed to reduce the wing loading at faster climb, resulting in lower deformations outside the clap-and-peel motion. The isolated effect of a body pitch angle is also studied. Here, the asymmetrical freestream direction results in larger asymmetries such as wing alignment with the freestream direction and reduced camber and even camber reversal during the upstroke. In forward flight the pitch angle is changed simultaneously with the flapping frequency and freestream velocity. Due to the non-linear properties the wing deforms not directly as a superposition of the individual effects. Deviations are mostly present in increased asymmetry in incidence angle, while the camber behaves more linear and the clap-and-peel motion also remains relatively unchanged. The torsional wave and recoil are here however reduced. Descending flight was also tested. Velocities below 1m/s result in relatively minor deformation changes, while faster descent leads to large flapping frequency fluctuations, making interpretation of the results impossible.

Accurate indoor localization is essential for autonomous robotic agents to perform tasks ranging from warehouse management to remote sensing in greenhouses. Recently Ultra Wideband (UWB) distance measurements have been used to estimate position and velocity indoors. These UWB-measurements are known to be corrupted by a varying bias. Besides, current estimation methods are not suitable for large areas with a low beacon coverage. The goal
of this thesis was therefore twofold. First, a simple bias model was proposed to reduce the influence of the UWB bias while still being implementable on a micro-processor. This model was shown to reduce the measurement error with 50% on validation data. Using this model, UWB-localization in a static beacon-configuration can be quickly improved. Second, an adaptation of the standard Moving Horizon Estimation (MHE) method was proposed that uses a time-window of range measurements to increase the robustness to outliers and is still real-time implementable on a micro-processor. This Moving Horizon Model Parametrization (MH-MP) does not estimate every state in the complete time-window, but only estimates an offset of the initial state in the window. An analysis of simulation data and data gathered in flight has shown that the proposed MH-MP outperforms the Extended Kalman Filter (EKF) in both the
position and velocity estimate and has a comparable computation time. Further research is necessary to investigate the possibility of estimating the UWB-bias model parameters online.