This thesis presents the design and evaluation of a comprehensive system for developing voice-based interfaces to support users in supermarkets. These interfaces enable customers to convey their needs across both generic and specific queries. While current state-of-the-art systems like GPTs by OpenAI are easily accessible and adaptable, featuring low-code deployment with options for functional integration, they still face challenges such as increased response times and limitations in strategic control for tailored use-case and cost optimisation. Motivated by the goal of crafting inclusive, personalised, and efficient conversational agents, this study advances on three fronts: 1) a comparative analysis of four popular off-the-shelf speech recognition technologies to identify the most accurate model for different genders (male/female) and languages (English/Dutch); 2) an assessment of the effects of personalised recommendations versus generic responses, using a blindfolded, counterbalanced within-subject experiment; and 3) the development and evaluation of a novel multi-LLM supermarket chatbot framework, comparing its performance with a specialized GPT model powered by the GPT-4 Turbo, using the Artificial Social Agent Questionnaire (ASAQ) in a counterbalanced within-subjects experiment and qualitative participant feedback. Our find-ings reveal that OpenAI’s Whisper leads in speech recognition accuracy across genders and languages, users significantly prefer personalised chatbots over the non-personalised counterparts and that our proposed multi-LLM chatbot architecture outperformed the benchmarked GPT model across all 13 measured criteria, including statistically significant improvements in four key areas: performance, user satisfaction, user-agent partnership, and self-image enhancement. The thesis concludes by presenting a simple method for supermarket robot navigation by mapping the final chatbot response to correct shelf numbers towards which the robot can plan sequential visits. This later enables effective use of low-level perception, motion planning, and control capabilities for product retrieval and collection. We hope this work encourages more efforts into using multiple, specialised smaller models instead of always relying on a single powerful model.

This paper proposes a novel framework that combines both planning and learning-based trajectory generation methods to handle complex robotic assembly tasks. The framework utilizes MoveIt! for planning large-scale reaching motions and Dynamic Movement Primitives (DMPs) for precise grasping and placing movements, with both methods integrated into a single system controlled by a behavior tree. An impedance controller is employed to ensure smooth and safe execution of the generated trajectories, particularly in scenarios that involve human interaction.

The proposed framework was evaluated within the context of the European Space Agency-funded Rhizome project, which focuses on off-earth habitat construction. The project involves assembling habitats using custom-designed Voronoi-shaped building blocks, which were also utilized in experiments to test the framework. The results showed that combining planning for large-reaching motions with DMPs for detailed movements effectively addressed the limitations of each individual method, delivering a flexible and robust solution to the challenges of robotic assembly.

In this work, we propose a method of processing patient input on discomfort level during robot shoulder physiotherapy into discomfort maps. These maps represent the patient's discomfort distribution throughout the range of motion of the shoulder, interpretable by both physiotherapists and robots. This method consists of three parts: the patient can input discomfort with a linear push-button; a collaborative robot arm is used to track the motion of the patient's shoulder; and audiovisual feedback of inputted discomfort is given to the patient and the therapist.
The method was validated in human factors experiments simulating shoulder physiotherapy sessions, where the subject is tasked with recreating a reference discomfort map through an auditory reference signal that emulates this discomfort. Here the robot also acts as the physiotherapist, moving the subject's shoulder. The signal is a beeping sound, whose rate scales with the discomfort intensity at the measured pose in the reference discomfort map.
We performed experiments with a total of 10 participants, demonstrating the viability of our method during patient-robot interaction. The results we collected also highlighted the presence of a time delay between the discomfort signal and the user input, and its effect on discomfort maps.

While Artificial Intelligence (AI) is geared towards automating tasks like writing and designing, the challenge persists in finding adequate human resources for tasks such as handling luggage in and out of airplanes or harvesting produce in greenhouses. Nonetheless, the demand to tailor robotic abilities to diverse scenarios, ranging from agriculture to household chores, necessitates a general-purpose morphology for the robot, such as a dexterous arm, along with sufficient sensory capabilities and intelligence to swiftly adjust to new situations.
Despite the prevalence of click-baiting videos shared online, current robot technologies have yet to address this requirement adequately. The primary obstacle hindering robot manipulators from effectively performing daily chores, aiding in supermarkets, and harvesting fruits from fields is the insufficient data available to construct a robust model of the world. Typically, autonomously exploring their surroundings and determining optimal strategies is considered unsafe and impractical.
A more effective approach to imparting knowledge to robots involves human supervision. Ideally, this entails interactive supervision where robots can seek clarification when uncertain about a situation, and humans can intervene when the robot’s actions are incorrect or fail to meet the required performance. Moreover, when receiving instructions or asking for them, the robot should quantify the confidence in the interpretation of the corrections. This thesis makes significant contributions to the field of interactive robot learning by introducing various uncertainty-aware methods. These methods facilitate enhancements in data efficiency during learning and safety during execution.
Before delving into the main contributions, Chapter 2 introduces the reader to the topic of Interactive Imitation Learning (IIL) and the different modalities that can be used to give feedback, from evaluative to corrective, underlying the importance of uncertainty quantification on the robot belief. For this reason, Chapter 3, introduces the foundations of the main function approximator used in this thesis, i.e. Gaussian Process (GP), to learn behaviors while quantifying uncertainties. The chapter highlights how a GP is trained given the evidence of the data and the corrections and how predictions of the mean and the variance of the actions are obtained. Particular attention is given to how GP models can be used for efficient updating and aggregation of online data and how to analytically estimate the uncertainty rate of change.
The proposed function approximator is first applied in Chapter 4. The presented machine learning framework allows the robot to learn complex manipulation tasks from interactive demonstrations. Essentially, the user needs to show a kinesthetic demonstration to the robot, i.e. dragging the robot around in a fully compliant modality to transfer their knowledge on a desired skill, e.g. cleaning a table or inserting a plug in a socket. The experiments highlight how the quantification and the rejection of uncertainties can be used to bring the robot always close to high-confidence regions. Moreover, the GP online model update is used to aggregate the corrections received from the user to reshape the learned attractor and the stiffness field. This ensures that the proper force is executed in the correct direction for instance when cleaning a table.
To extend the learning of a skill to the whole robot pose and gripper, Chapter 5 studies how to address this with GP and with the least amount of demonstrations and corrections. Moreover, the experiments focus on teaching human-like skills to robots by exploiting the possibility of giving incremental corrections. In particular, novice users, are asked to perform the picking task of objects in one fluid motion by teaching the complete pose and gripper behavior. The execution of the skill without any supervision is usually too slow or knocks the object down before closing the gripper. Nevertheless, after providing feedback, novice users were able to incrementally shape the robot’s velocity to perform the picking at non-zero velocity, without knocking the object and correcting for any delay in gripper dynamics.
However, learning skills only relying on the current robot’s Cartesian position can be a limitation since it cannot encode skills that entail overlapping, e.g. when approaching a goal and then moving back on the same trajectory. This motivates Chapter 6 which formulates a new trajectory encoding to teach single or bimanual manipulation skills while being safe around humans with constrained velocity and force actuation. The user study also investigates the effectiveness of giving kinesthetic corrections, i.e. by simply touching the robot, and validating this in teaching bimanual skills. Teaching two manipulators at the same time or correcting them using teleoperation devices can become overwhelming. Hence, the method explores adjusting movements interactively through kinesthetic perturbations rather than re-teaching skills entirely from scratch due to imprecise attempts.
Despite the successful applications of the proposed methods in single and bimanual motion skills, during task learning, the robot must not only master the motor aspect but also be attentive to the context, such as the object’s location or shape. This motivates Chapter 7, which emphasizes the generalization of acquired motor skills across various contexts. The proposed approach hinges on GP theory to acquire a non-linear transformation map from the demonstrated task space to the execution space while preserving and propagating uncertainties. Through experiments involving tasks such as pick-and-place operations, dressing human arms, and cleaning surfaces, it is demonstrated how the robot can generalize the execution by transforming the attractor, orientation, and stiffness policy to numerous new scenario configurations even with just a single demonstration of the skill.
In Chapter 8, the concept of task parametrization and uncertainty awareness is expanded to over-parameterizing the context, such as by tracking more objects than required. The proposed algorithm would prompt user attention when encountering ambiguity, like when multiple detected objects could be the goal of the skill. Decision ambiguity can be resolved by various feedback modalities, such as pushing the robot, moving it, or providing reward/punishment. A user study also highlighted the preference of novice users for not giving conventional kinesthetic demonstrations but only intervening when necessary.@en

Face recognition using lidar presents challenges arising from high dimensionality and data sparsity, especially at longer distances. This paper proposes a novel approach for face recognition via automotive lidar. The approach leverages a combination of deep learning and point cloud processing techniques. After identification of the facial point clouds, an alpha-shaped convex hull is employed for regional linearization, resulting in the creation of a depth image. This depth image is then fed to a convolutional neural network architecture, BasicNet, specifically trained for face recognition. The approach is evaluated on a dataset comprising 52 individuals acquired using two lidar sensors with different point densities. The individuals walked at distances ranging from 5 to 18 meters from the sensors. The approach achieves interesting results on this challenging dataset, thereby challenging the notion that lidar sensors are privacy-preserving.

This paper addresses the research question: “How can a human-robot team achieve co-learning, and interdependence in physically embodied tasks?”
A method has been developed that enables a human-robot team to co-learn the handover of an object from the robot to the human. Five design requirements were composed to address the challenges of human-robot co-learning in physically embodied environments. The method is based on a Q-learning algorithm that was adapted and extended to meet these requirements. An experiment was conducted with six participants. For every human-robot team, each design requirement was qualitatively evaluated. Interdependent co-learning was identified in three of the six teams. The limitation of the design, and how this method can be improved further, was discussed. The method, presented in this paper, demonstrates how human-robot co-learning and interdependence can be enabled in physically embodied tasks.

While defusing a bomb or performing a rescue mission with a teleoperated robot, grasping various objects is crucial. Despite being a routine activity, remote grasping is still challenging. It is difficult to apply an adequate grip force to avoid slippage and damage to an object. An additional challenge is controlling both motion and force at the same time during remote robot control (teleoperation). Therefore, this research presents a teleoperated semi-autonomous controller which assists the user with remote grasping by relieving the user from controlling the grip force. Our design enables the user (1) to control the position of the remote gripper while (2) the system controls the grip force autonomously. When the user grasps an object, the semi-autonomous controller maintains the grip force based on tactile feedback to prevent object slippage. For tactile feedback, our system uses a tactile sensor that can detect incipient slippage from deformations at the location of the contact. With two experiments, we show that the system can maintain an adequate grip force while being robust to external perturbations and input changes. Since this controller stably grasps objects while the user maintains control over the position of the remote robot, our method relieves the user and prevents object slippage

We propose a novel shared control interface that enables teleoperated teaching of both high-level decision-making skills and low-level impedance modulation skills using a single haptic device. In the proposed method, high-level teaching is achieved by repurposing the haptic device to remotely modify Behaviour Trees (BTs), allowing human operators to guide decision-making. Repurposing of the haptic device is achieved by exploiting its degrees of freedom for different functionalities. Low-level skill teaching involves an impedance command interface, that is used to command endpoint stiffness by manipulating a 3D virtual stiffness ellipsoid with the haptic device. Both teaching modes are connected: a newly demonstrated low-level skill appears in the BT at a user-specified index. Control is shared between the human and the autonomous system on a high- and low-level. At the higher level, the human can change the BT online, while ongoing execution of the low-level actions within behavior tree remains uninterrupted. During low-level teaching, shared control is implemented between the robotic motion skill and human-demonstrated stiffness. To provide a proof-of-concept and demonstrate the main features of the proposed interface, we performed several experiments in a teleoperation setup operating a remote shelf-stocker robot in a supermarket environment. A predefined BT encodes high-level decisions for a pick-and-place task. The impedance command interface is evaluated in a “peg-in-hole”-like task of placing a product on a cluttered shelf. Ultimately, the proposed interface can facilitate teleoperation-based Learning from Demonstration for the transfer of both high- and low-level skills in an integrated manner.

This thesis presents the research to design a successful high-accuracy, sub-millimetric registration method for an autonomous robot equipped to drill bone for cochlear implant (CI) surgery. Its performance, and thus success, is measured in accuracy, workload, usability and trust. While state-of-the-art (STOTA) research lacks the inclusion of human-robot interaction (HRI) related design requirements, this thesis demonstrates its significance. The HRI-focused design led to a successful multi-modal, multi-feedback admittance control, touch-based pair-point bone-anchored fiducial registration method.
A successful high-accuracy registration method is critical for image-guided robotic microsurgery as it contributes directly to its overall surgical precision. The inclusion of HRI-related requirements is concluded to be essential for the success of registration methods for surgical robots due to their inevitable involvement in a hospital workflow consisting of human teams and their lack of autonomous capabilities in all required tasks.
Gaining knowledge through theoretical and empirical research; utilising exclusion and scoring criteria to select the best specific registration method concept; and performing an extensive error- and human-factor analysis to determine the design implementations led to the design of a multi-modal multi-feedback admittance control, touch-based pair-point bone-anchored fiducial registration method. The design allows operators to execute a localisation task in collaboration with the robot. This design concerned a human operator physically guiding the robot end-effector to four bone-anchored fiducials attached to a skull-resembling setup to execute their localisation. The robot control was based on an admittance controller that enables the human to move the robot through the sensed interaction force at the robot’s end-effector. Consequently, if the operator exerted a forwarded force on the robot, it moved in that direction proportionally with the preset admittance parameters.
This guidance control contained five modes of control: (1) translational, (2) rotational, (3) fixed, (4) safety and (5) registration mode. Each mode allows the operator to control the robot as needed to achieve high accuracy, usability, trust, and a low workload. To switch between modes, the human operator utilised a foot-operated interface with several buttons, each dedicated to a specific function.
The translational mode had a fixed orientation and permitted free movements of various speeds along translational axes and was used for correctly arranging the end-effector to the fiducial centre in the x-, y- and z-axis. The rotational mode allowed orientation movement while preserving a fixed translational position and was employed for aligning the end-effector perpendicular to the fiducial centre. The fixed mode included a fixed orientation and translational position and was used for filtering out perturbations presented by the human operator. The safety mode allowed the robot to with-tract to a safety position and orientation in case of operator inactivity and was utilised to ensure patient safety. The registration mode allows for saving the end-effector position, hence localising the fiducials.
The feedback provided to the operator consisted of visual feedback, auditory feedback and a step-by-step graphical user interface (GUI). The visual feedback permitted colour coding of the modes of control and safety thresholds regarding force, torque and workspace. This was used to build operator awareness and control large errors introduced by operator input. The auditory feedback included a sound when localisation of the end-effector was completed and was utilised to form operator awareness and prevent timing errors. The GUI contained safety checklists, workflow guidance and operator performance feedback and was used to minimise operator variations.
To compare and benchmark our design, the baseline included a similar STOTA non-HRI focused single-mode no feedback admittance control touch-based pair-point bone-anchored registration method. This baseline method allowed a human operator to guide the robot end-effector in the translational axis and save fiducial localisation positions in cooperation with a second human operator. The comparison study involved a quasi-experiment of two groups with 13 and 14 participants, respectively. The groups executed repeated fiducial localisation and were compared on their system accuracy, workload, usability and trust.
The system accuracy was measured in fiducial localisation error (FLE) and target registration error (TRE) based on repeated measurements of each fiducial. The workload was estimated quantitatively in average operator force and qualitatively with a NASA TLX survey. Usability was calculated with the Computer System Usability Questionnaire (CSUQ) and System Usability Scale (SUS). The Trust Perception Scale-HRI measured trust. The design achieved the second-highest accuracy from all discovered STOTA registration methods. Furthermore, it reduced the operator workload and improved usability and accuracy compared to a non-HRI-focused design.
This thesis concludes that an HRI-focused symbiotic design provides a successful high-accuracy sub-millimetric registration method for robotic surgery as measured by accuracy, workload and usability.

To a future where humans and machines live together in harmony.



The comparison study in this thesis is performed in agreement with the TU Delft Human Research Ethics Committee (HREC).

In this work, we propose a method for monitoring and management of rotator-cuff tendon strains in human-robot collaborative physical therapy for rotator cuff rehabilitation. The proposed approach integrates a complex offline biomechanical model with a collaborative, industrial robot arm and an impedance controller. The model is used for computing rotator-cuff tendon strain as a function of human shoulder configuration, muscle activation and external forces. This subject- and injury-specific data is stored in \textit{strain maps}, which represent the relationship between the strains and shoulder DoFs. In our previous work, we implemented strain maps to preplan minimal strain, safe trajectories using two shoulder DoFs, and used the corresponding robot-mediated movement for passive trajectory following for healthy subjects. This work expands on that by implementing two novel functionalities: 1) patient-led movement, and 2) adding the third shoulder DoF and the corresponding control complexities, while still controlling for safe rotator-cuff tendon strains. For patient-led movement, we precomputed unsafe zones for each strain map by clustering and fitting ellipses to the clusters. These unsafe areas with increased risk of (re-)injury are then used to set the impedance control parameters and reference pose for real-time biomechanical safety control. By linearly interpolating between strain maps, smooth and safe movement of the third shoulder DoF is added. The resulting robot control torques guide the patient away from unsafe, high strain shoulder poses in real-time during patient-led movement. The proposed method has the potential to improve the safety, Range of Motion, and muscle activity that the patients receive through robot-mediated physical therapy. The main advantage of this approach is that the patient is free to use and explore their full shoulder RoM, while the robot controls and manages biomechanical safety in real-time. To validate the proposed method, we performed two experiments showcasing two novel functionalities, and a third experiment as proof-of-concept displaying the full method.

Walking is an essential part of almost all activities of daily living. Depending on the situation, different gait patterns can be observed, e.g., moving around the house, performing different sports, or even in case of injury. Even though the gait has been analyzed thoroughly for many decades, there are still some unexplored aspects that require more insight, especially those related to the influence of various parameters on the optimality and diversity of gait patterns. Many gait trajectory optimization strategies have been proposed in literature, however, most of them focus merely on optimizing for one metric (e.g., energy efficiency or joint torque). In this study, a multi-metric gait optimization framework is proposed, simultaneously accounting for joint torque, fatigue, and manipulability. To that end, 45 gaits, varying in stride length, step height, and walking speed, were recorded in a motion capture experiment, together forming a solution space of dynamically stable and physiologically feasible gaits, for the proposed optimization framework. Specific user needs (gait requirements) can be accounted for by adjusting the optimization weights, after which brute-force optimization is applied to either analyze the gait within the collected subspace or select the optimal gait with respect to desired parameters. Results are presented for a baseline case (with all optimization weights set to one), which can be used as a tool for gait analysis, in particular giving insights into specific aspects of the gait, e.g., joint loading, long-term performance, and capacity to sustain ground reaction forces (GRFs).

In this research, we examined the occurrence of skill mutation when propagating a collaborative sawing task from robot-to-robot. We conducted this research, to gain insight into this mutation to understand the generation of potentially beneficial or dangerous skills.
Thirty propagation steps in simulation were conducted per experiment, each consisting of one expert robot teaching a novice (learner) robot. To explore what influences mutation, different external factors were changed, such as the maximum stiffness of the robots, the base position of the robots, the friction coefficient of the object and saw, and the period span of one sawing movement. The robots were controlled using a hybrid force/impedance controller, with the impedance part responsible for the sawing movement. The goal of the skill propagation was to teach the novice robot the impedance controller inputs (desired trajectory and stiffness), which is implemented through a three-staged learning process. In stage one, the desired trajectory was learned by encoding the measured trajectory using DMP and LWR, in stage two the stiffness was learned by encoding the computed stiffness, and in stage three the novice robot became an expert, able to collaboratively execute the task.
The results showed that the skill varied over the different propagation step, therefore proven the existence of skill mutation. It was found that the biggest mutations were caused by a phase lag, overshoot on the desired trajectory, differences in joint states, and reaching torque limits. It was also found that environmental boundaries limited the mutations. By comparing the results of the different propagation steps of both different and the same conditions, it was that the mutations were not reproducible. This is a result of not being able to fix all external factors. We also identified the benefits (skill useful for different settings or different tasks, and energy efficiency) and dangers/drawbacks (high forces and skill becoming useless for initial task) of the mutation.

To generalize the use of robotics, there are a few hurdles still to take. One of these hurdles is the programming of the robots. Most robots on the market today employ position control, with a set of controller parameters tuned by an expert. This programming is quite expensive, only suited for a single task, in a single configuration, and not interaction safe. This thesis tries to solve these problems, by introducing Position And Stiffness Teaching with Interactive Learning (PASTIL) and History Aware PASTIL (HA-PASTIL), a novel interactive way of learning scalable variable impedance policies. The system is able to learn both positional reference trajectories and stiffness trajectories at the same time. PASTIL and HA-PASTIL learn these policies from positional corrections applied by a human teacher, through physical human robot interaction (pHRI). For the measurement and extraction of these corrections only the proprioception sensors of the robot are used, so no force/torque sensors are required. To learn from these corrections, the intention of the teacher is estimated, by segmenting the correction space in three parts. Each of these three parts correspond to a set of update rules for the policy, that fit the intention of a correction in that segment. In this thesis, the proposed algorithms are validated through a series of experiments with sample tasks, and compared with baseline algorithms. The main conclusions from these tests are that PASTIL and HA-PASTIL, as introduced in this thesis, outperform the baseline algorithms on task performance for all tasks and that the learned stiffness makes a positive contribution to task performance. This means that the algorithms proposed here allow for simple systems, with only proprioception sensors, to be instructed by users, instead of experts. This makes it possible for robotics to be applied at lower cost, with less expertise needed to program and operate. These algorithms, however, still have some aspects that could use further research. The most important example is that they are not yet tested on an actual robot, with physical human robot interaction. There is still quite some work left to do, but the proposed algorithms might pave the way for more, and better, algorithms that aim to learn force control behaviour form only positional corrections.

We are living in an aging society, which is putting an increasingly heavy strain on our healthcare system. As people age many become less mobile, leading to loneliness and a deteriorating health. Subsequently elderly often end up in nursing homes; an experience which is unpleasant as well as expensive. Assisting the elderly with robotic devices can help increase their mobility, therefore reducing healthcare costs and increasing elderly satisfaction. This thesis follows up on the master thesis of R. Berci-Hajnovics, where she sets out to design a novel concept for an assistive, motorized shopping trolley aimed at reducing the physical burden of carrying groceries over uneven terrain. In the current thesis, a control system for the proposed assistive shopping trolley is designed with the goal of attenuating disturbances like road inclination and added mass on the trolley’s dynamic behavior. Predictability of its behavior is considered as well in order to gain the confidence of the elderly user. An analysis of potentially suitable control types for implementation in the assistive shopping trolley has been performed, wherein their characteristics are compared on several different aspects relating to their predictability and disturbance attenuating behavior. Based on this analysis, a controller implementing a so called Disturbance Observer (DOB) is deemed best suited for the job. A DOB uses a dynamic model of the controlled plant to estimate the influence of present disturbances on the plant dynamics. This estimation can consequently be used to produce an opposing force, thereby canceling the effects of the disturbances. Furthermore, a DOB includes a filter which provides additional capabilities, such as robustness to uncertainty within the model and filtering of high-frequency noise in the data to further increase its performance. A DOB is most suitable for the current cause due to its overall adequate performance with respect to the imposed criteria, as well as its relatively high disturbance attenuation accuracy due to its ability to use the plant’s dynamics in making an estimation of the disturbances at play. Following the results of the analysis, this thesis proposes the implementation of a controller including a DOB for application in the assistive shopping trolley. In its design, a model of the trolley dynamics is being used, which provides a basis for the controller’s internal dynamic model. This model is furthermore used as a representation of the trolley plant in a stability analysis of the trolley-controller closed-loop behavior. The model is based on the dynamics of an inverted pendulum on wheels, with an additional constraint on the position of the trolley handle to represent the interaction with the human user. By neglecting minor, nonlinear dynamic effects such as variations in the trolley’s orientation and air drag, the general model is simplified to a linear equation. The controller’s internal model is constructed to represent the trolley’s nominal (desired) behavior, determined to be that of an empty trolley on a flat road, by substituting corresponding nominal values in the parameters of the general model. Subsequently, criteria for robust stability of the trolley-controller’s closed loop behavior are determined using a H_∞-approach; In this approach the system’s behavior is analyzed given the ‘worst-case’ uncertainties. Furthermore, the system’s performance with respect to disturbance estimation and noise filtering is analyzed using its loop transfer function. The insights gained from these analyses are used in the design of the controller and to make recommendations for future work. The obtained controller is implemented in the Robotic Operating System (ROS) framework; an open-source robotics platform which makes use of a decentralized system of processes, simplifying the creation and sharing of complicated software structures across a wide variety of applications. For this purpose, the controller is subdivided into multiple elements, each performing a specific function. These elements, called ‘nodes’, are submitted to a variety of unit tests to verify their proper implementation. The controller’s performance is put to the test in simulation using ROS’s accompanying simulation engine, ‘Gazebo’. The controlled trolley’s acceleration error is compared to that of a regular trolley in several scenario’s, where different types of disturbances are applied. The results show that the controller indeed reduces the effects of disturbances on the trolley’s behavior; the controller is able to recognize the trolley’s changing behavior due to an encountered disturbance and it correctly attenuates its effect by ordering the required compensatory motor torque. More research is required with respect to the effect of uncertainty within the model dynamics to ensure stability and controllability of the trolley system. Moreover, the closed-loop system response should be tested in the presence of various types of noise to determine how well the current results apply when the controller is used in a real-life scenario.

Tele-impedance can increase interaction perfor- mance between a robotic tool and unstructured/unpredictable environment during teleoperation. However, the existing tele- impedance interfaces have several ongoing issues, such as long calibration times and various obstructions for the human oper- ator. In addition, they are all designed to be controlled by the operator’s arms, which can cause difficulties when both arms are used, as in bi-manual teleoperation. To resolve these issues, we designed novel foot-based tele-impedance control method inspired by the human limb stiffness ellipse modulation. The proposed mechanical interface is based around a circular disc and a foot pressure sensor that controls orientation and size/shape of the stiffness ellipsoid, respectively, by the operator’s foot. We evalu- ated the circular disc interface control method in an experimental study with 12 participants, who performed a two-dimensional drilling task in a virtual environment. The results show the ability of the operator in controlling the proposed interface to dynamically adapt to task instruction and environment. In addition, a comparison with low and high constant impedance control demonstrate a superior interaction performance of the proposed interface.

The general approach to generate collision free motion in a constraint environment is to use path planners, which demand a known environment and potentially fail otherwise. Learning from Demonstration (LfD) can be used instead to teach the robot unknown parts of the environment, such as a goal deviation or an unforeseen obstacle. The general approach is to train a model offline and expect it to perform well afterwards. Problems arise however when the model is trained insufficiently or unknown variations have occurred in the environment, which demand for refinement of the model. In online learning the operator is allowed to refine a predicted trajectory during execution time, where the state-of-the-art methods focus on kinaesthetic teaching. The contribution of this research is the development of a teleoperated online learningmmethod, where the operator can make refinements by moving a haptic stylus (Phantom Omni) in the desired direction. By doing this, a force is felt proportional to the magnitude of refinement. After creating such a refined trajectory, it is used to update a condition dependent probabilistic trajectory model. The proof of concept was shown on a 2D example and on a simulated robot that shows that we can adapt an initial model when unknown variations occur and that the method is able to deal with different object positions and initial end effector poses. To show if other people can use the method, a human factors experiment is performed, comparing the developed method against three other methods on how much time it takes to successfully adapt a model (refinement time) and on the perceived workload. Two different parameters are varied, which are the teaching device (stylus or keyboard) and the learning mechanism (online or offline). The expectation was that both online with stylus has the lowest refinement time and workload, but the results show that only online has a significant improvement over offline methods (p = 7.94 × 10^−12 and p = 0.000512 respectively). This is explained by the fact that only small corrections have to be made and in a maximum of three degrees of freedom (DoFs). No significant difference was found between keyboard and stylus (p = 0.755 and p = 0.302 respectively). An explanation for this is that this is task, person and implementation dependent. The recommendations are to evaluate the proof of concept on the real robot and to extend the method with orientation refinement, such that more complex tasks can also be dealtwith. We hypothesize that with these tasks the combination of online with stylus does perform the best.