Airlab, a collaboration between TU Delft and Ahold Delhaize, is developing Albert, a robot tailored to work in a complex supermarket environment. Key to Albert is a product detection and classification module that tells it what products to grasp and where they are located in a shelf. Albert’s existing YOLO‑based product detector a significant issue: Adding new products without re‑training the whole model is impossible. Especially in a dynamic supermarket environment with an ever‑changing stock, the latter is a major issue.

This problem will be the main focus of this paper and is addressed through few‑shot learning, which predicts similarity between query and target products. This simplifies adding new products to just supplying new target images. Few‑shot learning also requires significantly less data to train on. In supermarkets with 300.000 different products, requiring only a few images per product is a major advantage. For this reason, this paper aims to deploy a few‑shot model to classify products as either the target class or non‑target class for Albert’s picking task and defines the following research question: “What few‑shot classifier can identify products in a supermarket environment, is able to detect non‑target classes, and meets the requirements of deployment on a robotic platform like Albert best?“

This paper first analyses the potential of using TRIDENT and P>M>F, two state‑of‑the‑art few‑shot models, for deployment on Albert, and evaluates them on the requirements of this paper. P>M>F performs better on all requirements, which makes it the preferred model for Albert. However to work well, it still requires adjustments. Its inference time is still too high to work on Albert and it cannot classify query images as not the target product.

For this reason, this paper uses P>M>F’s two key ideas to construct Product‑ProtoNet, a new Albert‑suitable few‑shot model: 1) Using a good pre‑trained feature extractor; and 2) Comparing query images to a set of classes and matching only to the likeliest. P>M>F uses a ProtoNet model for classification that essentially does this; Like ProtoNet, Product‑ProtoNet constructs class prototypes from one or multiple examples of class images. Product‑ProtoNet then uses a sigmoid classifier to predict if query images have the same class as those prototypes. It compares query images to a set of similar class prototypes(helper prototypes) and classifies it as the likeliest. Product‑ProtoNet uses a ViT pre‑trained with DINO to extract image features. To bring down inference time, Product‑ProtoNet computes product prototypes before deployment.

With an accuracy of 99.1% on product classes seen during training and 99.8% on novel classes in a realistic supermarket setting, a low inference time of 2.89 ms and a memory usage lower than 4GB, Product‑ProtoNet is the only model that passes all requirements of this paper. When deployed on Albert Product‑ProtoNet successfully guides Albert to the right product in 97% of attempts. This makes Product‑ProtoNet the only few‑shot classifier that can identify products in a supermarket environment, is able to detect non‑target classes, and meets the requirements of deployment on a robotic platform like Albert.

Spatial object detection and environmental understanding are fundamental aspects of autonomous driving in mobile robots. In this work, a stochastic approach to 2D LiDAR and stereo camera sensor fusion for object detection is presented. By tracking LiDAR clusters 360° around the robot and continuously updating knowledge of their corresponding object types using Bayesian inference when they are within the camera’s view, uncertainties in the detection algorithm are addressed. Additionally, localisation and sensor uncertainties are accounted for by using Gaussian distributions to model the locations of entities. By estimating the state of these entities, dynamic objects can be tracked throughout the environment, while remembering their type from when they were last visible in the camera’s view. The results demonstrate correct linking of camera detections with LiDAR clusters, with an average positional inaccuracy of objects of 0.12m in simulation, and 0.25m across different experiments on the robot, as well as the ability to track humans even when they move out of the camera’s field of view with an average tracking error of 0.33m in simulation.

This thesis investigates the stability and robustness of the Lyapunov Actor-Critic (LAC) algorithm in comparison to the widely-used Soft Actor-Critic (SAC) algorithm. Motivated by the need for reliable and robust control systems capable of operating in dynamic and unpredictable environments, the research initially aimed to explore LAC's performance in an effort-controlled robotic manipulation task. However, the focus shifted to a reproduction study of Han et al. 2020 after identifying key discrepancies in the implementation details, hyperparameters, and undocumented aspects of their study.

The primary goal was to validate the reproducibility of Han et al. 2020's results and assess the role of critical parameters, such as the α₃ stability constraint, in the performance and stability of the LAC algorithm across various simulated environments. By reimplementing and rigorously testing the algorithms, our study confirmed that the α₃ parameter significantly impacts the performance and stability of the LAC algorithm, particularly in complex environments where maintaining high stability is essential for successful operation. The results demonstrated that a more conservative α₃ value, which tightens the stability constraint, improves performance and stability but also increases the time needed for the agent to adapt and stabilize its policy, highlighting the need to balance stability and adaptability in scenarios demanding both high performance and strict stability. Despite some small differences, the study confirmed the reproducibility of Han et al. 2020's findings, demonstrating the potential benefits of the LAC algorithm compared to SAC in terms of stability and robustness. Furthermore, the study showed that other hyperparameters, such as network architecture, training length, and horizon length, greatly influence the performance and stability of the LAC and SAC algorithms, underscoring the importance of precise hyperparameter tuning, transparent reporting, and consistent experimental configurations in reproducing and validating results in reinforcement learning research.

By validating the stability of the LAC algorithm across different simulated environments, and through the development of detailed guidelines for key hyperparameters and a robust codebase, this research provides a solid foundation for extending the algorithm's application to more complex tasks, such as effort-controlled environments. These contributions facilitate reproducibility and pave the way for future research into the LAC algorithm's ability to operate in dynamic and unpredictable environments, ultimately supporting the safe deployment of learning-based controllers in real-world systems where stability and reliability are paramount.

These days, people see more and more applications for drones, including monitoring rainforests to protect plant and animal species. However, drones face challenges when navigating through the dense and cluttered vegetation of the forest. These environments necessitate advanced autonomous detection and navigation to make the drone traverse robustly and fly safely. In addition, the forest brings extra challenges, such as blocked signals for GPS localisation, remote control, and remote supervising.

In this thesis project, a drone is designed, built, and programmed to navigate autonomously in the rainforest with complete onboard computing and no GPS localisation. This 500-gram drone is being extensively tested and optimized in real forest conditions, and a dataset is being created from its autonomous flights to simulate various configurations of the path-planning algorithm. The results of these simulations on this dataset are then used for thorough research on how the algorithm can downscale to smaller systems and how this affects performance.

By using the results of this research on downscaling, a 100-gram drone is built and programmed to fly in forest conditions with complete onboard computation. Challenging on this small-size drone is the use of low-quality lightweight sensors and processor. The processor only weighs 10 grams, and the depth camera weighs 8 grams. Unique on this small drone is the 3D path planning fully computed onboard and the implementation of a new type of depth camera.

As robotics will play a crucial role in the future of our modern societies, the need for advancements in the field is more pronounced than ever. While robots are already present in industrial settings, they are noticeably absent from dynamic environments. Dynamic environments are characterized by other moving agents, such as humans, varying tasks and high safety requirements. With the aim to deploy robots in such environments, Trajectory Generation (TG) becomes crucial. TG approaches aim to compute sequences of control commands that bring the robot from its current configuration to a desired goal state while avoiding collisions with obstacles and itself. Thus, it is directly placed between task planning, the problem of defining what high level tasks should be executed in which order, and control, the problem of executing motor commands. A good solution to TG must be fast, to cope with changes in the environment, flexible to different goal definitions, and it should promote safety. Most advancements in the field of robotics in TG focus either on manipulators or on mobile robots. However, the combination of both systems seems inevitable for deployment in human-shared environments.

TG for mobile manipulation is usually formulated as an optimization problem of a finite time horizon. This approach is known as Model Predictive Control (MPC) and is widely used in the field of autonomous driving thanks to its feasibility and stability guarantees. In Chapter 4, we present a method to bring MPC to mobile manipulation. The method formulates the TG problem for the entire kinematic chain and relies on Free Space Decomposition (FSD) for collision avoidance. This leads to reasonable control frequencies of 10Hz independent on the amount of collision obstacles in the environment. Importantly, this approach allows for coupled motion of the mobile base and the manipulator. This is beneficial in situations where synchronization of the two subsystems is crucial, such as opening doors or moving obstacles around. Despite these simplifications on the environment representations, computational costs limit the applicability of MPC to mobile manipulation as motion is not considered truly reactive and different components, such as goal attraction and collision avoidance, are not easily separable.

A recent novel approach to receding horizon control is the formulation as a purely geometric problem. Early successes in this direction, including Cartesian Impedance Control (IC) and Artificial Potential Fields (APF), led to the formulation as sets of dynamical systems on smooth manifolds in the configuration space. The framework of Optimization Fabrics (fabrics) unifies such ideas, offers stability guarantees in static environments, and results in highly reactive behavior similar to simple low-level controllers. This framework relies on non-Riemannian geometry to shape a smooth manifold of the configuration space with individual behaviors, such as collision avoidance, joint-limit avoidance, and goal attraction. In Chapter 5, we present a generalization of fabrics to dynamic environments. We refer to the resulting framework as Dynamic Fabrics (DF). The generalization uses time-parameterized manifolds to integrate moving obstacles and time-parameterized reference trajectories. The latter is especially important for long-horizon TG that may exhibit local minima. Importantly, Chapter 5 shows that the dynamic component of DF is required when coping with moving obstacles. As repulsive forces in fabrics are proportional to the approaching speed of obstacle and robot, collision avoidance in a pseudo-static fashion is not sufficient when the robot is moving slowly. Finally, we deploy the general framework of DF to several real-world settings showing the applicability of the framework to mobile manipulation. First, we present a way to integrate non-holonomic constraints into the framework. Despite loosing formal guarantees on convergence, the method is shown to be the natural extension to wheeled mobile robots characterized by non-holonomic constraints. Second, Chapter 6 presents a symbolic implementation of fabrics to achieve higher control frequencies. Symbolic implementations are possible because the framework of fabrics is based on differential equations of second order, for which a closed-form solution exists. For changing environments, obstacles states are then only concretized at runtime. Additionally, we show symbolic hyperparameters can be tuned automatically to achieve expert-level tuning performance. Third, to overcome high requirements on the perception pipeline, Chapter 7 integrates different implicit environment representations into the framework. Using Signed Distance Field (SDF) and FSD for example is widely used in mobile robotics when formulating TG as MPC. We show that the same representations can be used in fabrics while achieving faster solver times. Finally, Chapter 8 deploys a mobile manipulator controlled by fabrics in a supermarket. Dexterous manipulation is programmed using learning-from-demonstration, with fabrics as the underlying encoding. That allows to teach rather than program complicated behaviors while maintaining properties on collision avoidance.

This thesis presents insights into aspects of motion planning, advances the framework of fabrics for TG, and compares it extensively to the more commonly used method of MPC. Through the ideas presented in this thesis, we hope to encourage the usage of geometric properties of robotic systems deployed to human-shared environments. This approach does not only provide reactive TG but also may act as a compact encoding of trajectories for learning-based methods in the future.
@en

Object pushing in robotics has numerous applications, but it often relies on room-bound object tracking systems such as Motion Capture (MoCap) for accurate object pose acquisition. Such systems limit the potential use scenarios, since they add complexity and cost and require expansion of the sensor infrastructure for expanding the operational areas. To address these limitations, we propose a framework that integrates object pushing and vision-guided object pose estimation using a monocular RGB camera that is placed on the pushing agent. By leveraging the temporal coherence of the scene, we accelerate the 2D object detection stage that is common in the field of object pose estimation. Our contributions include the development of a novel framework that continuously pushes an object while simultaneously tracking its pose. We incorporate dynamics by efficiently using a Model Predictive Path Integral controller’s predictive model to predict the object location in the forthcoming image, resulting in an extreme reduction of computational cost for 2D object detection, without sacrificing the accuracy of the pose estimation. Our framework achieves real-time object pushing with accurate object pose estimation, which we demonstrate with real-world experiments. Furthermore, we provide a novel object pose estimation dataset in the widely used BOP format, specifically for robot planar pushing with an on-board camera. Our approach pushes towards eliminating the need for room-bound sensor systems, expanding the potential use cases. By considering temporal coherence and scene dynamics, our framework challenges recent object pose estimation methods that fully process each image as uncorrelated individuals, while providing a promising solution for real-time object pushing.

Automated vehicles represent an exciting advancement in transportation, offering a range of benefits that have the potential to revolutionize how we travel. They can improve safety, efficiency, accessibility, and sustainability, holding promise for transforming our cities and communities. However, generating safe, comfortable, and efficient motion plans, especially in interactive scenarios like forced lane merging, remains a significant challenge in the field.

Lane merging is a pivotal skill for automated vehicles, as it frequently involves changing lanes to reach a destination. For instance, when approaching an intersection, a vehicle might need to merge into a specific lane beforehand to execute a turn later. Traditional pipelines in automated driving typically decouple prediction and planning. They assume perfect upstream prediction and generate robust motion plans to avoid collisions with multi-modal predictions. However, in dense traffic conditions, conservative planning might hinder the ego vehicle from merging effectively, resulting in it becoming stuck. This underscores the necessity of combining prediction and planning, a concept we term interaction-aware planning algorithms.

The first major contribution of this thesis is an efficient game-theoretic behavior planner that captures interactions under different behavior modes. In this approach, we represent the behaviors of the vehicles as actions in a matrix game and select the Nash equilibrium to capture their mutual influence. To generate the cost of the action pairs, we model the merging process as a gap selection process and evaluate the trajectories generated by interactive models. The effectiveness of the proposed planner is validated in the high-fidelity CARLA simulator.

In the real world, human drivers may not always adhere rationally to the equilibrium of a game model. They could choose a behavior mode different from the game theory solution. Therefore, it might be more beneficial to consider different motion modes simultaneously, rather than favoring the "most likely" one while neglecting the others.

For this purpose, we also explore the usage of Branch Model Predictive Control (B-MPC) in this thesis. By predicting the motion of the surrounding vehicle as a scenario tree, the B-MPC approach can generate a trajectory tree as a motion plan. By executing only the root node, the ego vehicle can consider different future scenarios simultaneously and plan contingency motions. We further extend the B-MPC approach by incorporating interactive policies, using different solving schemes, and including collision avoidance constraints that consider the orientations of the vehicles. The effectiveness of these proposed methods was validated through experiments conducted in a handcrafted lightweight simulator.

Overall, this thesis focuses on developing interaction-aware planning methods to facilitate safe, successful, and comfortable lane merging scenarios. Nevertheless, the major limitations lie in the modeling error and the potential long-tail issues. To address these challenges and further improve motion planning, future work could explore data-driven (learning-based) approaches that leverage real-world driver behavior to generate more informed and adaptable motion plans.

Visual inspection of liquid medicine containers for contamination and defects is mandatory and crucial to ensure their safety for injection. This document presents research and development of three modules of the Visual Particle Inspection Subsystem (VPIS), an automatic inspection subsystem with the task of detecting and classifying particle contamination in liquid medicine containers. The proposed VPIS comprises three modules for which research and development is performed separately: the backgrounds subtraction and segmentation module, the classification module, and the tracking module.

For the background subtraction and segmentation module, a solution is proposed with filtered temporal background modelling and locally adaptive threshold segmentation. This solution works on the assumption that moving objects such as particles and bubbles are sparse and do not occur more than twice in a pixel in twenty frames. Therefore, a representative incomplete background cluster can be obtained by removing the two lowest or highest pixel values depending on the mode of illumination. Next, the average value of the background cluster is used for background subtraction and the standard deviation of the background cluster is used to determine a locally adaptive segmentation threshold. The solution has been positively evaluated for detection of low contrast objects, insensitivity to disturbances, processing time, and parameter configuration. Additionally, an add-on solution is proposed that makes it possible to detect objects in the challenging region near the rubber stopper of a syringe.

For the classification module, out of four candidate classification methods a Convolutional Neural Network (CNN) is chosen for the classification of single object detections. The CNN classifier achieves an accuracy of 0.93 and is used as a baseline classifier for the rest of this research. Next, with three exploratory research questions, research is performed into opportunities and pitfalls for this classification problem summarised below.
-It is found that certain handcrafted features correlate to the classification accuracy of a detection. A method is proposed to predict the classification accuracy of detections and filter out detections that cannot be classified reliably. Compared simple filters that filter out detections based on a single feature such as area or total contrast, the proposed method should achieve a higher subsequent classification accuracy and reject less detections.
-With the simplest classification strategy that is often used for inspection, a container will be rejected if a single detection is classified as a particle. It is shown through simulation that this classification strategy is not effective for this classification problem. This is as the large number of bubble detections in a clean container would result in a high false container rejection rate.
-Two multi-detection classification strategies are proposed that use multiple detections for classification. Median voting classification uses results from the tracking module to classify multiple detections of the same object. Multi-positive classification does not use tracking results but requires multiple detections to be classified as particles. Simulation results indicate that with both strategies, the classification accuracy for containers can be drastically improved. It is theorized that the best results can be achieved with median voting classification...

Simulation environments are useful for a wide range of applications and their functionalities continue to improve every year. The aim of this thesis project was to create a simulation environment with high levels of realism and assess its capabilities through the use case of generating distributed drone traffic rules.

This thesis project was executed at the company Almende in cooperation with its sub-company DoBots. The distributed drone traffic rule generation use case was a contribution to the ADACORSA project which Almende takes part in. The goals of the thesis were the following. A simulation environment incorporating elements of realism was created using the software tools Almende has experience with. The capabilities of this environment were tested through the said use case. According to company requirements, this process was realised using an incremental complexity approach, and it utilized machine learning techniques. It was expected that the incremental complexity approach would help speed up the learning process of more complex scenarios, while machine learning would provide the benefit of smart automation and impartiality to solutions. For this process, a model was also devised for creating such rules, incorporating the incremental complexity approach and a standard machine learning technique. Finally, the benefits of the incremental complexity approach were also assessed.

The four building blocks of the implemented simulation environment were Gazebo, PX4, ROS2 and the iris drone model. The code was built upon the OpenAI ROS software package, with extensive modifications. The main work done during the implementation consisted of multiple parts. The original software package was cleaned of unnecessary elements, the PX4 software and the iris drone model were integrated into the environment and the simulation speed was increased. The whole setup was migrated from ROS to ROS2. The use of multiple drones in the same environment was also solved. An implementation of the deep Q-learning algorithm, which was selected as the learning method, was added. Different scenarios were created, as well as some logging and visualization tools. Additionally, the system was set up to run in docker containers and was highly parameterized for future use.

Different elementary traffic scenarios were implemented in the simulation and then executed. Based on these the following points were determined. It was concluded that the created simulation environment was well-fitted for running scenarios for traffic rule generation. It was also suitable for executing the scenarios using the incremental complexity approach, although its usability also depended on how the neural networks of the scenarios were set up. The environment was also feasible to work with machine learning from an engineering viewpoint, however, it did not perform well for its scientific research. This was due to not having statistically well-supported results, which was caused by the long time it took to execute a scenario. The benefits of the incremental complexity approach were also examined. For the simple case tackled in this project, it was found that the incremental complexity approach provided no significant advantages with regard to learning speed.

This master thesis introduces Hierarchical Active Inference Control (HAIC) as a control method for nonholonomic systems. This method only requires tuning of a minimal number of hyperparameters and has a relative low computation load. HAIC is based on recent research done in the application of the neuroscientific theory of Active Inference for robot control. The hierarchical aspect of this method introduces two layers of Active Inference Control (AIC) and the introduction of an additional decision hyperparameter. The top layer AIC is designed such that it takes the nonholonomic constraint into account. This causes HAIC to be able to control a nonholonomic mobile robot to a three-dimensional reference state in comparison to regular AIC which is unable to. The selection of the introduced hyperparameter gives a trade-off between robustness against noise present in the measurements and possible error in
the y-dimension of the local reference frame.

Convergence of a two-wheeled differential-drive mobile robot to a reference state using HAIC is demonstrated both by simulation and physical experiment. Simulations are done for a large range of different starting states. These simulations show the ability of HAIC to converge the mobile robot for any starting state. It also confirms that a higher value for the decision parameter causes a larger error. Another group of simulations is done investigating the new
hyperparameter introduced with HAIC. These simulations check the ability to converge the system when different noise levels are present in the measurements. It is concluded that tuning of the newly introduced hyperparameter needs to accommodate the noise present in
the measurements. Where the higher the noise, the higher the value of the hyperparameter is needed for convergence.

A physical experiment is done where the robot is controlled to five different reference states. Data obtained from this experiment shows the ability of HAIC to control a physical nonholonomic system. Like the simulations, it is also shown that a correct value is required to set the newly introduced hyperparameter. Additionally, the physical experiment shows that HAIC is not robust when disturbances can cause sudden large value changes in the measurements.

Autonomous robots are often successfully deployed in controlled environments. Operation in uncontrolled situations remains challenging; it is hypothesized that the detection of abstract discrete states (ADS) can improve operation in these circumstances. ADS are high-level system states that are not directly detectable and influence system dynamics. An example of a typical ADS problem that is used in this thesis is that of a wheeled robot driving through puddles of mud that, when entered, alters the velocity of the robot. When the robot is in such a puddle, it is in an ADS 'mud', and when it is not, it is in an ADS 'free'. ADS can be indirectly inferred through the analysis of lower-level data such as the velocity of the robot.

The goal of this thesis is to design a general abstract discrete state estimator (ADSE) operating with limited prior knowledge. An ADSE is a hierarchical system for detecting changes in ADS. The ADSE should be general; applicable to multiple ADSE problems. The ADSE should further operate under limited prior knowledge: only assuming that the amount of ADS and the ADS that describes the regular operation are known. The basis for the ADSE designed in this thesis is a Gaussian hidden Markov model (GHMM), a hidden Markov model enhanced with Gaussian emissions.

Randomly generated experiments are done on a simple but general ADSE problem. Two unsupervised learning methods derived from Expectation Maximization are evaluated, namely Baum-Welch (BW) and forward extraction (FWE). FWE is introduced in this thesis and is a simpler implementation of Viterbi extraction, leveraging assumptions of ADSE to in theory gain computational efficiency. We found that both BW and FWE exhibit superior performance compared to a likelihood-based baseline estimator when the maximum score of the learning curve is considered. When the final score is considered, in some cases, FWE displays a deteriorating learning curve, resulting in worse final scores compared to the baseline. Furthermore, it was found that the lower the overlap coefficient (therefore the less similar the ADS), the higher the maximum reached score. It was further shown that BW exhibits better convergence than FWE to the true model parameters. Besides this, FWE obtained comparable or in some cases even superior scores compared to BW.

In general, from the results, the diversity of the experiments conducted, and the assumptions made we can conclude that the GHMM can be a general method for an ADSE with limited prior knowledge. To quantify the suitability of the GHMM for ADSE, further research should include the evaluation of different ADSE methods on the same problem. There exists a tradeoff between the lower computational cost FWE and the more stable but more computationally intensive BW learning. Therefore, future research can include a combination of these methods. Other extensions include extending the GHMM to a Gaussian mixture hidden Markov model to allow for the modeling of more complex distributions, or the application to multiple states or a changing environment.

In the literature, neural network compression can significantly reduce the number of floating-point operations (FLOPs) of a neural network with limited accuracy loss. At the same time, it is common to manually design smaller networks instead of using modern compression techniques. This thesis will compare the two approaches for the object detection network YOLOv7. YOLOv7 can run in real time on a desktop GPU. For edge GPUs a smaller version, called YOLOv7-tiny, was manually designed by the authors of YOLOv7. This thesis answers the question: Can a state-of-the-art compression of YOLOv7 achieve higher accuracy than YOLOv7-tiny at the same number of floating-point operations?
First, two state-of-the-art compression methods are selected and compared on YOLOv7-tiny. Then the best performing method, GBIP, is used to compress YOLOv7 till it has the same number of FLOPs as YOLOv7-tiny. From the experiments it is determined that GBIP is not able to achieve higher accuracy than YOLOv7-tiny at the same number of FLOPs.

In recent years, visual Simultaneous Localization and Mapping (SLAM) have gained significant attention and found wide-ranging applications in diverse scenarios. Recent advances in computer vision and deep learning also enrich visual SLAM capabilities in scene understanding and large-scale operation. However, despite remarkable performance in these fields, most visual SLAM frameworks are designed with the static world assumption. Thus, they often confront challenges in dynamic environments, manifesting reduced localization accuracy, tracking failures, and restricted generalization.

To investigate the impact of moving objects in dynamic indoor environments, we first benchmark representative visual (dynamic) SLAM approaches, complemented by robustness assessments for preliminary insights. During this process, we adopt challenging sequences from GRADE, an ideal platform for simulating dynamic indoor scenes. Notably, the mainstream of dynamic SLAM methods employs detection or segmentation techniques as solutions. To explore the correlation between detector accuracy and overall SLAM performance, we integrate a series of trained YOLOv5 and Mask R-CNN models, each with varying accuracy levels, into dynamic SLAM systems. Subsequently, we evaluate these configurations on the TUM RGB-D sequences. Contrary to common intuition, the experiments indicate that more accurate object detectors do not necessarily lead to improved visual SLAM performance. This benchmarking process also illuminates several inherent limitations of current dynamic SLAM techniques, underscoring the imperative for further advancements.

Building upon these insights, we introduce DynaPix SLAM, an innovative visual SLAM system for dynamic indoor environments, where participation of visual cues (e.g., features) is weighted based on per-pixel motion probability values. Our approach consists of a semantic-free pixel-wise motion estimation module and an improved pose optimization process. In the first stage, our motion probability estimator employs a novel static background differencing method on both images and optical flows to identify moving regions. These probabilities are then incorporated into the map point selection and weighted bundle adjustment for backend optimization. We evaluate our DynaPix SLAM and its variant, DynaPix-D, in comparison with ORB-SLAM2 and DynaSLAM. These assessments are performed on both TUM RGB-D and GRADE sequences, with additional tests on the static versions of the GRADE ones. The results demonstrate that DynaPix SLAM consistently outperforms the other methods, showcasing reduced localization errors and longer tracking durations across various scenarios.

Object detection is one of the hottest topic right now and is a fundamental concept which determines to future of autonomous driving. There are hundreds of papers rolling out every day trying to improve the performances of these detectors by creating more complex models to increase the performance by a very little amount. The problem with current state-of-the-art approach is that the uncertainties of the model and the datasets are not being quantified. This is where the gap in the research lays as the uncertainties are not taken into account but also are not being used in the model itself. This research paper answers the question on how the uncertainties of a 3D LiDAR-based object detector can be quantified and used in 3D object detectors. This paper showcases that regarding semantic segmenta- tion and 3D object detection, that the multi-frame detectors have a great possibility of incorporating the uncertainties not only to improve the performances of the models but also to improve the quantification of the uncertainties. A key contribution of this paper is that it has shown that the best techniques for quantifying the uncertainties is to make use of a combination of MC dropout and variatonal inference to approximate the uncertainties, which can then be used to represent the spatial uncertainties in the detected objects. This research showcases that there is yet to be a paper which concisely uses these spatial uncertainties for multi-frame detectors. Furthermore, in order to create such a model, some of the most fundamental datasets have been analyzed (Kitti-360, Kitti, nuScenes, etc.) to show that the nuScenes, Waymo and Kitti-360 datasets are suitable for this kind of detection.

This thesis focuses on closed-loop product grasping from supermarket shelves. The case is studied where the robot is in front of a shelf in an Albert Heijn supermarket and is tasked to pick a desired product from that shelf. Enabling a robot to achieve the product-picking task, however, is challenging. While many other robotic picking methods are centered around table-top environments, the complex geometry of supermarket shelves presents a challenge in itself. Additionally, the Albert Heijn supermarket is a dynamic environment, where other agents can change the shelves, move products, and can introduce lighting changes. Where the table-top environment allows other methods to pick objects with an open-loop controller, approaching the supermarket environment with a closed-loop picking strategy can be beneficial in overcoming the challenges introduced by the dynamic environment. Therefore, this thesis proposes a product grasping pipeline, where the goal is to discover what combination and optimization of the required robotic skills results in a system that enables the robot to consistently and robustly perform the product-picking task. To close the loop, in-hand position-based visual servoing is used that enables the robot to account for detection mistakes as it picks a product. The three robotic skills that are required for the product-picking pipeline are product detection, product grasp pose estimation, and product tracking. Because of visual servoing, each of these robotic skills must run in real-time. The product detection is achieved by pre-training a YOLOv6 object detector on the SKU-110K dataset and fine-tuning it to the new Albert Heijn Supermarket dataset. The Albert Heijn Supermarket dataset is created to detect 36 products in the supermarket, where the challenges of distinguishing similar products and detecting relocated products are included. To enable detections during visual servoing, the distance, directions toward the shelf, and lighting are varied. The product grasp poses for the suction cup can be estimated using a plane fit on the estimated pointcloud for each product. The pointcloud of the product is estimated by randomly sampling the depth data from the product. Each product pose is then tracked through time via Kalman Filtering, to enable temporal reasoning about the products. Because of this, the grasp pose of the desired product can be refined as the manipulator moves toward the shelf. The proposed system achieved success rates of 90\%-100\% during experiments on a real robot with a suction cup gripper. While robust picking on a set of 36 products has been achieved, exploring a wider variety of product shapes with other robotic grippers is a compelling research direction to overcome the challenge of picking oddly shaped products. Furthermore, a separate few-shot-learning classifier for the product classification might be used to overcome the challenge of adding new products to the inventory or product re-branding. Next, considering other picking scenarios in the supermarket, like picking from hooks or refrigerated shelves with doors, is important for deployment as well. Finally, interacting with humans and reacting to human behavior during the picking process to ensure safety is another crucial challenge that must be overcome to move toward the integration and deployment of robotics in supermarkets.

This thesis is inspired by Active Inference to contribute to its improvement in the Robotics work field. However, the results and applications of this thesis are useful in a broader perspective, namely in any field that makes use of derivatives and the forecasting of a time-series signal. The goal of this study is to determine a new approach for calculating the derivatives using the finite difference technique and forecasting. The finite-difference technique is based on the Taylor expansion. The Taylor expansion uses derivatives to calculate a future signal sample, while the finite difference technique uses signal samples to estimate derivatives. The accuracy of the estimation can theoretically be increased using more signal samples and choosing signal samples that are closer to the signal sample of interest. For these reasons, the central method is interesting as it maximizes both of these properties. The central method uses future signal samples for the sample of interest which are not available in an online setting. Future values can be forecast through for example statistical models and predictive coding. Statistical models such as the auto regression method are the golden standard but cannot learn in an online fashion. Predictive coding can learn in an online fashion and research found a promising branch utilizing neural network frameworks. PredNet came out as an interesting model due to its state-of-the-art results and flexibility. The research question in this thesis is as follows: can predicting future data points by use of PredNet help improve the accuracy of the generalized coordinates?
Two models of PredNet are trained on data obtained from random commands from a simulated Jackal robot. One is trained in an online fashion and one in an offline fashion. Its predictions are compared against two other forecasting methods: copying the input and the auto regression method. The predictions are then used with the finite difference method to estimate derivatives. Four methods are used: a method using as little past data as possible, a method using as much past data as possible, a method using as little centralized data as possible and a method using as much centralized data as possible. Derivatives are determined on the same simulated Jackal data and are compared against a derivative obtained from a central method using as much data as possible that all come from the true signal.

Results show no forecasting difference between the two types of PredNet models, but there are differences in training speed: training in an online fashion is not fast enough to keep up with the data stream. This suggests that the quality of neural networks is good enough for online applications, but that more work needs to be done on optimizing neural networks for online learning to make it a practical solution. However, the PredNet predictions are not as accurate as the auto regression method. This means that neural networks can still learn from the auto regression method when used for forecasting time series. An example of improving neural networks is making hybrid models.
Using the PredNet forecasts for estimating derivatives did not increase the accuracy of the derivatives meaning the predictions are not accurate enough. Besides that, using more samples, either from the past or the future, did not increase the derivative estimate either. Most of this outcome can be attributed to the choice of data set which contained many transitions between commands which could not be predicted by the PredNet models or be seen by the finite difference method.
Suggestions for improvements are to focus the training of a model on either the transition period or the period in between, using more modern models and/or using different methods for estimating derivatives.

This work concludes that neural networks show promise for online applications, that training a model requires a specific data set or a complex enough model to utilize a more general data set, and that using more samples with the finite difference method is not guaranteed to provide more accurate estimates of derivatives.

Active Inference control is a novel control method based on the free energy principle, which combines action, perception and learning [1][2]. The first Active Inference controller showed promising results on a 7-DOF robot arm for a pick and placing task, however it took nearly six seconds to converge which is too slow [2]. This thesis aims to reduce the convergence time of an Active Inference controller. Therefore an Active Inference controller that is used to control the velocity of a Jackal robot was developed. It was compared to the standard differential drive Controller and it was found that the standard controller outperformed the Active Inference controller on both rise and settling time. A state space model was derived to obtain a better understanding of the Active Inference controller, for this model the robot was modelled as a point mass. This assumption was confirmed by a step response test of the robot, placing it both on the box and on the ground. It was found that both behaved similar, and thus the wheel ground interactions and internal dynamics of the robot did not affect the convergence time. The state space model was used to find three methods to reduce the convergence time of the Active Inference controller. As a first method the update frequency of the entire controller was increased, for the second method only the update frequency of the inner belief update loop was increased. The last method used an increased update frequency of the belief update loop and the parameters of the system were tuned. The methods were tested using step response tests and a square wave test, first in a simulated Gazebo environment and later with the actual robot. It was found that the third method reduced the converging time of the Active Inference controller, and reduced it from 0.88s to 0.51s. The improved controller was also tested against the differential drive controller outperformed it. It was found that due to the increased update frequency the tuning parameters could be changed to a wider range of values, this resulted in a shorter convergence time.

Mobile robots are getting more common in warehouses, distribution centers and factories, where they are used to boost productivity. At the same time, they are moving into the everyday world, where they need to operate in uncontrolled and cluttered environments. In order to extend the set of tasks that a robot can autonomously accomplish in such environments, we can equip mobile bases with a pushing skill. Possible use cases are delivering packages by pushing objects to a goal location, or clearing a path by pushing movable obstacles out of the way.

This thesis considers pushing with a non-holonomic mobile robot in cluttered environments. We develop a learning-based method, based on ensembles of probabilistic neural networks for learning the object’s dynamics. The models capture the inherent stochasticity of pushing from the variability of friction. In addition, the ensembles can capture the uncertainty that arises from a lack of data, so we can reason about which regions to avoid during the pushing and do not come into unrecoverable states.

The model is combined with a Model Predictive Path Integral (MPPI) controller. Each timestep the controller optimizes for a finite time horizon, which allows us to recover from slight model inaccuracies. The MPPI controller is capable of handling complex cost functions, which allows us to include the estimated uncertainty from the model to avoid uncertain regions. For obstacle-aware pushing, we provide the MPPI controller with a costmap grid, that contains the free pushing space, which the robot and object can use to maneuver to the goal location. Both the robot and object are constrained to keep within the free pushing space, which is constructed using LiDAR measurements.

We verify the method in simulation and compare it to two baseline methods: the state-of-the-art adaptive pushing controller proposed by Krivic et al. [31] and a pushing policy trained with the model-free reinforcement learning baseline Soft Actor-Critic (SAC). All three baselines reach comparable success rates in simulation of 100%, 98% and 96%, respectively. Furthermore, the learning-based MPPI shows improvements of 50% and 23% in accuracy compared to the baselines. The results are consistent for pushing in cluttered environments, where we show an increase of accuracy by 43%, compared to Krivic et al. [31].

Lastly, we validate the approach in a real-world setting, where the robot and object are tracked with a motion capture system. The approach has an overall success rate of 94% and performs successful pushing in a cluttered environment.