In the field of robotics, consider the following problem scenario: In a robot environment, a simple robot must push objects to reference places while figuring out which objects can be pushed, what the best manipulation strategy is, or which objects are static and cannot be pushed. The problem scenario can be decomposed into three research topics which individually have received much attention from the research community; learning object dynamics [8, 37], Navigation Among Movable Objects (NAMO) [7, 13, 21, 23] and nonprehensile pushing [2, 5, 25, 41, 42, 43]. A combination of these three topics could lead to improvements in planning, execution time, and reasoning, but it has not been explored in the literature. This thesis proposes a robot framework that combines these three research topics. This framework comprises of three key components: the hypothesis algorithm, the hypothesis graph, and the knowledge graph. The hypothesis algorithm computes a hypothesis on how to relocate an object to a new pose by computing possible action sequences given certain robot skills. In doing so, the hypothesis algorithm creates an hypothesis graph that encapsulates the structure of the action sequences and ensures the robot eventually halts. Once a hypothesis is carried out on the robot, information about the execution, such as the outcome, the prediction error, the type of controller used and other metrics, are stored in the knowledge graph. The knowledge graph is populated over time, allowing the robot to learn, for instance, object properties and then refine the hypothesis computed to increase task performance, such as success rate and execution time. A new planning algorithm is proposed that can detect a when a path is blocked by an object, the hypothesis algorithm relies on the newly proposed planner to generate action sequences and to free blocked paths. This planner extends the double tree optimised Rapidly-exploring Random Tree algorithm [7]. The planner constructs a configuration space for an object and is provided with starting- and target pose for that object. The planner then converts these poses to points in configuration space to then search for a path connecting the starting configuration to the target configuration. For the new planner, objects are initially classified as “unknown” and can later be categorized as either “movable” or “unmovable”. The object type information is then used when constructing the configuration space for the newly proposed planning algorithm. Configuration space consists of the conventional free- and unmovable- (or obstacle) space and the newly proposed unknown- and movable space. To carry out the investigation, a mobile robot in a robot environment with movable and unmovable objects is created. The robot is given a task that involves relocating a subset of the objects in the robot environment through driving and nonprehensile pushing. The task can be broken down into individual subtasks that consist of an object and a target pose. Planning for a push or drive action occurs with the newly proposed planning algorithm that, if successful, completes a given task and populates the knowledge graph with learned object information. Information that can be used to determine which objects to manipulate and what strategy performs best to manipulate a specific object. In an effort to develop a robot framework that combines these three topics, a framework is created that shows improved task execution as a result to experience gained in the robot environment. The proposed framework performs equivalent or better compared to the state-of-the-art frameworks that are specialized in only two out of three research topics [47]. It can be concluded that the framework partly combines the three topics because learning system models with a system identification module is moved to the future work section. Instead, the proposed method selects the best available control and system model combination in the set of available control and system model combinations.

Autonomous robots are often successfully deployed in controlled environments. Operation in uncontrolled situations remains challenging; it is hypothesized that the detection of abstract discrete states (ADS) can improve operation in these circumstances. ADS are high-level system states that are not directly detectable and influence system dynamics. An example of a typical ADS problem that is used in this thesis is that of a wheeled robot driving through puddles of mud that, when entered, alters the velocity of the robot. When the robot is in such a puddle, it is in an ADS 'mud', and when it is not, it is in an ADS 'free'. ADS can be indirectly inferred through the analysis of lower-level data such as the velocity of the robot.

The goal of this thesis is to design a general abstract discrete state estimator (ADSE) operating with limited prior knowledge. An ADSE is a hierarchical system for detecting changes in ADS. The ADSE should be general; applicable to multiple ADSE problems. The ADSE should further operate under limited prior knowledge: only assuming that the amount of ADS and the ADS that describes the regular operation are known. The basis for the ADSE designed in this thesis is a Gaussian hidden Markov model (GHMM), a hidden Markov model enhanced with Gaussian emissions.

Randomly generated experiments are done on a simple but general ADSE problem. Two unsupervised learning methods derived from Expectation Maximization are evaluated, namely Baum-Welch (BW) and forward extraction (FWE). FWE is introduced in this thesis and is a simpler implementation of Viterbi extraction, leveraging assumptions of ADSE to in theory gain computational efficiency. We found that both BW and FWE exhibit superior performance compared to a likelihood-based baseline estimator when the maximum score of the learning curve is considered. When the final score is considered, in some cases, FWE displays a deteriorating learning curve, resulting in worse final scores compared to the baseline. Furthermore, it was found that the lower the overlap coefficient (therefore the less similar the ADS), the higher the maximum reached score. It was further shown that BW exhibits better convergence than FWE to the true model parameters. Besides this, FWE obtained comparable or in some cases even superior scores compared to BW.

In general, from the results, the diversity of the experiments conducted, and the assumptions made we can conclude that the GHMM can be a general method for an ADSE with limited prior knowledge. To quantify the suitability of the GHMM for ADSE, further research should include the evaluation of different ADSE methods on the same problem. There exists a tradeoff between the lower computational cost FWE and the more stable but more computationally intensive BW learning. Therefore, future research can include a combination of these methods. Other extensions include extending the GHMM to a Gaussian mixture hidden Markov model to allow for the modeling of more complex distributions, or the application to multiple states or a changing environment.

The problem of assisting users in comprehending the robotic scenario information in a retail setting has been studied. To design the system, an integrated ontology composed of several IEEE standard ontologies and a labelled property graph (LPG)-based ontology modified from the Web Ontology Language (OWL)-based ontology was proposed to symbolize information in the robotic environment. Then, a knowledge graph (KG)-based chatbot was developed to provide natural language interaction with users. A case study in a retail setting was designed, and the results were analyzed. The effectiveness of our designed system has been experimentally validated in both static and dynamic scenarios, with at least 1.5 times improvements.

This thesis proposes the novel Behaviour Tree Update Framework (BTUF) for the initial construction and continuous incremental adaptation of Behaviour Trees (BTs) for applications in Learning from Demonstration (LfD) frameworks to create complex robot behaviours associated with Activities of Daily Living (ADL) without requiring the user to have a programming or engineering background. BTUF implements several methods towards that end. Automatic generation of the fundamental structure of BTs within BTUF allows for easy human operation of the framework's Text-based User Interface (TUI). Saving and loading of constructed trees facilitates easy expansion and reusability of constructed trees. By expanding upon an initial base behaviour, seemingly simple behaviours can be adapted to facilitate novel instances thereupon, increasing the complexity and functionality of the constructed tree over time. Experimental validation in the form of a user study has provided proof-of-concept within simulation and has given insight in the initial overall performance and general system acceptance of BTUF. Future work is recommended for further validation and improvement of the proposed framework to one day realise a real-life application within healthcare robotics.

The autonomy of mobile robots has been greatly improved in recent decades. For these robots, the field of search and rescue is of particular interest. This thesis introduces a new method to let a mobile robot (Spot by Boston Dynamics) explore and search for victims in unknown environments. Existing methods include coverage, which aims to fully cover the environment as efficiently as possible. Exploration methods place more emphasis on gaining knowledge of the environment quickly, but do not actively search for victims. A new method based on active inference is introduced with the aim of combining exploration and exploitation behaviour within one framework. The active inference model is based on a graph representation of the environment, formulated as a POMDP. The framework is built up by incrementally more difficult cases. The first case allows a mobile robot to navigate a known graph to search for victims. The second case assumes a partially unknown graph. Uncertainty about the existence of unvisited nodes is included in the predictions. The final case adjusts the model to the agent's point-cloud- and camera sensors. The framework is then used in a simulation environment, showing how it can be implemented in real-world scenarios. To do so, the active inference framework is combined with techniques from information gain exploration. This thesis shows that active inference can be used in large unknown environments to carry out search and exploration.