In this thesis, an autonomous multi-robot system for indoor exploration in limited network environments is proposed. The specific use case is search and rescue where the operators must have access to the most up-to-date information, necessitating the requirement for communication maintenance. This requirement is satisfied in a novel way by using signal strength measurements collected by the robots to update the network model, thus reducing the overly conservative nature of current approaches. The network model chosen is an adaptation of existing path loss models as they balance complexity and performance given the information available during multi-robot exploration.

The proposed system consists of a central server with multiple robots which perform tasks that are readily distributable, whereas the central server performs complex iterative optimizations such as merging the local maps and running task assignment. While fully distributed architectures offer theoretical benefits such as scalability and robustness, in practice, these benefits are currently not achieved due to the complexity of splitting iterative optimizations such as mapping and task assignment among multiple nodes. This results in distributed architectures requiring more time and bandwidth to solve these problems, which is why this work implements a central architecture. While iterative optimizations are more efficient when performed centrally, scalability is limited due to the branching of possible options during task assignment. To combat this, an action space formulation, called an action graph, is proposed that is unique to each robot and reduces the number of actions by merging similar ones.

Both the dynamic network prediction model and the action graph formulation are shown to hold promise by experimentation. However, more work is needed before real-world use is feasible.

Learning new concepts is a difficult task for autonomous robots. These robots can adapt to changes in the situations. To adapt to a situation, they should be able to determine the usefulness of objects around them. The usefulness of objects is highly dependent on situational context, making pre-programming of adaptation behaviour to all possible situations difficult. Automatic learning of this information by the robot from its own observations during deployment is a more feasible approach. We encode the usefulness of objects in logical statements that symbolise regularities in observations of objects in a Markov Logic Network. This network forms the basis from which the usefulness of newly observed objects or situations can be inferred by a robot. Each logical statement in the network has an associated weight that indicates how strong the robot believes the statement is true. These weights can be adjusted over time based on new observations made by the robot, strengthening its beliefs or creating a new belief for an unknown object in a process called cumulative learning. We extend the Markov Logic Network framework with a cumulative learning algorithm called MLN-CLA. This algorithm contains several different knowledge updating strategies that handle conflicts. We demonstrate the ability of MLN-CLA to learn the usefulness of unknown objects over time. In addition, we demonstrate the abilities of MLN-CLA to incorporate new logical statements to better capture information about objects and situational contexts. Together, these two abilities enable robots to autonomously adapt to changing situations during deployment.

This work builds upon the Gaussian Belief Propagation (GBP) stack, utilizing it as the core framework for distributed multi-robot exploration missions. The GBP stack’s key strength lies in its single-factor graph representation of competencies such as planning, map consensus, and task allocation, making it a powerful tool for intelligent and collaborative robotic behavior. However, previous applications of GBP stack were limited to open environments without obstacles. To address this limitation, we extend the GBP stack by representing the environment as a navigational graph, allowing for task allocation and global path planning in more complex, obstacle-filled environments. This enhancement integrates seamlessly with the single-factor graph approach, enabling both navigational tasks and symbolic tasks, such as opening doors or performing inspections, to be efficiently distributed among a heterogeneous fleet of robots. A feature absent in earlier versions. Simulation results demonstrate improved task coordination, exploration efficiency, and adaptability to varied tasks, showcasing the potential of this extended framework for more challenging multirobot exploration scenarios.

Quadrupedal robots possess the ability to move freely in the world and perform a variety of actions that would be unsafe or impractical for humans to perform. In the SNOW project, a quadrupedal robot is tasked with aiding firefighters in rescue missions during house fires by locating humans and assessing their health. Pushing away obstacles that cannot be circumvented otherwise is one of the many capabilities a quadrupedal robot should possess to be of most use in such missions. We develop a proof of concept by solving two problems sequentially: which stance to take on prior to the push and how to perform the push.

The process of stance selection starts with generating a certain amount of stances. Stances are generated starting from a preselected stance appropriate to the goal location and deviating from the 12 joint angles with a normal distribution. All generated stances are ran through a number of filters, which rely on solving for the forward and inverse kinematics of the robot. These filters check if the initial position is sensible and balanced and if the projected final position is close to the goal and balanced. The inverse kinematics are solved using Adaptive-Network-Based Fuzzy Inference Systems (ANFIS), which results in accurate estimations within a time frame that can be used in real-time applications. The final stance is selected by comparing the total displacement of all joint angles per stance, where the lowest total displacement is considered optimal.

The push is controlled by a nonlinear model predictive controller. We strive for a stable push, where the contact between the pusher and the object sticks, by keeping the movement of the end-effector within the motion cone. The motion cone denotes all twists the object can have without slipping at the contact with the pusher and is constructed using the limit surface to model the interaction between the object and the support surface and the generalized friction cone to model the interaction between the pusher and the object. We find that the motion cone predicts stick and slip with an accuracy slightly higher than 80%. Our controller steers accurately to all goals that lie within the motion cone and moves the object with a twist on the edge of the motion cone if the goal location lies outside of the motion cone. The robot remains balanced throughout the pushing motion in the vast majority of cases, but is more at risk of tipping over when pushing heavier objects.

This master thesis introduces Hierarchical Active Inference Control (HAIC) as a control method for nonholonomic systems. This method only requires tuning of a minimal number of hyperparameters and has a relative low computation load. HAIC is based on recent research done in the application of the neuroscientific theory of Active Inference for robot control. The hierarchical aspect of this method introduces two layers of Active Inference Control (AIC) and the introduction of an additional decision hyperparameter. The top layer AIC is designed such that it takes the nonholonomic constraint into account. This causes HAIC to be able to control a nonholonomic mobile robot to a three-dimensional reference state in comparison to regular AIC which is unable to. The selection of the introduced hyperparameter gives a trade-off between robustness against noise present in the measurements and possible error in
the y-dimension of the local reference frame.

Convergence of a two-wheeled differential-drive mobile robot to a reference state using HAIC is demonstrated both by simulation and physical experiment. Simulations are done for a large range of different starting states. These simulations show the ability of HAIC to converge the mobile robot for any starting state. It also confirms that a higher value for the decision parameter causes a larger error. Another group of simulations is done investigating the new
hyperparameter introduced with HAIC. These simulations check the ability to converge the system when different noise levels are present in the measurements. It is concluded that tuning of the newly introduced hyperparameter needs to accommodate the noise present in
the measurements. Where the higher the noise, the higher the value of the hyperparameter is needed for convergence.

A physical experiment is done where the robot is controlled to five different reference states. Data obtained from this experiment shows the ability of HAIC to control a physical nonholonomic system. Like the simulations, it is also shown that a correct value is required to set the newly introduced hyperparameter. Additionally, the physical experiment shows that HAIC is not robust when disturbances can cause sudden large value changes in the measurements.

In this work, we address the challenges of employing robots in the Search-and-Rescue (SAR) domain, where they can benefit rescue workers to quickly obtain Situational Awareness (SA). Missions with autonomous mobile robots are heavily dependent on environmental representations. Representations have been steadily increasing in the richness that they can capture, in addition to geometry we can now represent objects and their interrelations. These richer environmental representations, such as the 3D scene graph, have provided an opportunity to integrate representations with prior knowledge to make them more actionable and improve the SA they provide. The use of such representations for planning is limited. In previous work, the main approach was to augment and extend the scene graph to enable traditional symbolic planning. The limitations of these methods are that they are offline, scale poorly, and require full observability, making them unsuitable for the SAR domain. The main contributions of this work are as follows. First, we propose the behavior-oriented situational graph, a data structure that integrates data-driven perception with prior knowledge following a novel situational affordance schema. This schema connects situations with robot behaviors and mission objectives, allowing for autonomous mission planning. Second, we propose an efficient method to obtain task utilities from the proposed behavior-oriented situational graph through planning. Finally, we propose an exploration component to discover and select tasks online in dynamic environments that are potentially partially observable. This work is evaluated in several simulation scenarios, showing improved efficiency in mission completion compared to offline methods for the specific SAR domain. Finally, the methods are implemented and tested in a real-world scenario using a mobile Spot robot, showing its effectiveness in practice.

During the whole history of Computer Science as we know it, the end goal has been to solve problems faster and more efficiently than us humans. Computers came to be to perform repetitive tasks we do not want to or do not have time to perform ourselves. At some of them, such as simple mathematical calculations, they were and they are indeed better and faster than a human being. With the appearance of computers, some effort was also put in the study of the human brain and the idea of building machines with similar principles of operation: The idea of emulating conscience has always been attractive to humankind.

The area surveillance problem is the problem of surveying a known or an unknown area with the main purpose of detecting objects. This thesis will tackle the problem of how to employ a group of mobile-sensors for surveying an unobstructed area in an optimal manner. The mobile-sensors should make use of their onboard computers and they should iteratively compute their waypoints until they successfully surveyed the area. To solve the area surveillance problem in an optimal manner, the mobile-sensors should be able to coordinate their actions. The Distributed Constraint Optimization Problem (DCOP) framework will be employed to define the area surveillance problem. In the DCOP, the mobile-sensors can define their optimal position for the next discrete time step by communicating with the other mobile-sensors that participate in the problem.
For solving this DCOP, the Distributed Pseudo-tree Optimization Procedure (DPOP) will be utilized. The DPOP is a complete solver that can find the optimal solution of a DCOP in a decentralized manner. However, the main limitation of DPOP is that the size of the largest message that the mobile-sensors will have to exchange is space-exponential in the induced width of the pseudo-tree. Considering that the mobile-sensors have to use their onboard computers for solving the problem, exchanging huge size of messages is not desired. To overcome this limitation of DPOP, a new extension, known as the MST-DPOP will be presented in this Thesis. The MST-DPOP makes use of the Maximum Spanning Tree (MST) algorithm to reduce the size of the largest message that the mobile-sensors have to exchange. Employing MST along with DPOP can bound the size of the largest message and the required computations for constructing the utility messages. However, the new extension cannot guarantee that its solution will be the optimal. The experimental results shows that the MST-DPOP is able to define a solution with an average error less than $2\%$. Moreover, the MST-DPOP requires on average around $1$ discrete time step more than the DPOP to solve the area surveillance problem. Consequently, given that the MST-DPOP overcomes the high memory requirements of DPOP, it is preferred for solving the area surveillance problem.

Reinforcement learning (RL) is an area of Machine Learning (ML) concerned with learning how a software-defined agent should act in an environment to maximize the rewards. Similar to many ML methods, RL suffers from the curse of dimensionality, the exponential increase in solution space with the increase in problem dimensions. Learning the hierarchy present in underlying problems, formulated using the Markov Decision Processes (MDPs) framework, may exploit inherent structure in the environment. Using the hierarchical structure, an MDP can be divided into several simpler semi-MDPs (SMDPs) having temporally extended actions. The solutions of smaller SMDPs can then be re-combined to form a solution for the original MDP. The methods for Hierarchical Reinforcement Learning (HRL) explore ways to break down the original problem into SMDPs while providing several opportunities for state and temporal abstractions. A novel algorithm for learning this hierarchical structure of a discrete-state goal-oriented Factored-MDP (FMDP) is proposed in the thesis work taking into account the causal structure of the problem domain with the use of Dynamic Bayesian Network (DBN) model. The proposed method autonomously learns the state and temporal abstractions in the problem domain and constructs a hierarchy of SMDPs using them. Such a decomposition results in decreasing the problem state dimensions to be considered for solving each SMDP and, hence, reducing the computational complexity induced due to increased dimensionality.

Autonomous underwater vehicles (AUVs) are unmanned vehicles that operate underwater. These vehicles can be used for various operations, including scanning the ocean floor in order to search for objects. In such operations, the AUV navigates close to the ocean floor, using its sonar systems to map this ocean floor. Before an AUV is launched on a search operation, usually very little is known about the underwater conditions at the search location. Currently, the AUV’s coverage path is often defined before the start of the operation. However, the coverage path depends on the quality of the sonar images, which in turn depend on the unknown and varying underwater conditions. Therefore, it is unlikely that the predefined coverage path is optimal. In this final thesis project a coverage path planner is developed, which autonomously adapts the AUV’s coverage path to the changing underwater conditions during an operation. Since the radio waves carrying the global positioning system (GPS) signal do not travel far in seawater, the AUV needs to keep track of its location using an inertial navigation system (INS). Unfortunately, the error on the location measurements obtained from an INS grows over time, causing the AUV’s believed trajectory to differ significantly from the AUV’s true trajectory. As a result, parts of the search area might be believed to have been covered, while these parts have in reality not been visited. Taking the AUV’s uncertain trajectory into account thus increases the complexity of the coverage path planning problem. In order to keep track of which parts of the search area have been covered, and to which extent, a coverage map of the search area is maintained during an operation. This coverage map is constructed such that it takes the uncertainty on the AUV’s location measurements into account. Three different coverage path planners have been developed that use this coverage map to autonomously plan the AUV’s coverage path during an operation. The three different coverage path planners have been implemented in MATLAB, and they have been evaluated in a simulation framework developed by TNO. The simulation results show that all three coverage path planners consistently achieve the required coverage of the search area, in a scenario where no large changes in the underwater conditions occurred. Furthermore, one of the coverage path planners has been evaluated in three scenarios where large changes during the operation did occur. Halfway during these scenarios, either the strength of the underwater currents increased, the required coverage of the search area increased, or one of the AUV’s sonar systems broke down. In all of these scenarios, the coverage path planner successfully adapted the coverage path such that the required coverage of the search area was still achieved.