While Artificial Intelligence (AI) is geared towards automating tasks like writing and designing, the challenge persists in finding adequate human resources for tasks such as handling luggage in and out of airplanes or harvesting produce in greenhouses. Nonetheless, the demand to tailor robotic abilities to diverse scenarios, ranging from agriculture to household chores, necessitates a general-purpose morphology for the robot, such as a dexterous arm, along with sufficient sensory capabilities and intelligence to swiftly adjust to new situations.
Despite the prevalence of click-baiting videos shared online, current robot technologies have yet to address this requirement adequately. The primary obstacle hindering robot manipulators from effectively performing daily chores, aiding in supermarkets, and harvesting fruits from fields is the insufficient data available to construct a robust model of the world. Typically, autonomously exploring their surroundings and determining optimal strategies is considered unsafe and impractical.
A more effective approach to imparting knowledge to robots involves human supervision. Ideally, this entails interactive supervision where robots can seek clarification when uncertain about a situation, and humans can intervene when the robot’s actions are incorrect or fail to meet the required performance. Moreover, when receiving instructions or asking for them, the robot should quantify the confidence in the interpretation of the corrections. This thesis makes significant contributions to the field of interactive robot learning by introducing various uncertainty-aware methods. These methods facilitate enhancements in data efficiency during learning and safety during execution.
Before delving into the main contributions, Chapter 2 introduces the reader to the topic of Interactive Imitation Learning (IIL) and the different modalities that can be used to give feedback, from evaluative to corrective, underlying the importance of uncertainty quantification on the robot belief. For this reason, Chapter 3, introduces the foundations of the main function approximator used in this thesis, i.e. Gaussian Process (GP), to learn behaviors while quantifying uncertainties. The chapter highlights how a GP is trained given the evidence of the data and the corrections and how predictions of the mean and the variance of the actions are obtained. Particular attention is given to how GP models can be used for efficient updating and aggregation of online data and how to analytically estimate the uncertainty rate of change.
The proposed function approximator is first applied in Chapter 4. The presented machine learning framework allows the robot to learn complex manipulation tasks from interactive demonstrations. Essentially, the user needs to show a kinesthetic demonstration to the robot, i.e. dragging the robot around in a fully compliant modality to transfer their knowledge on a desired skill, e.g. cleaning a table or inserting a plug in a socket. The experiments highlight how the quantification and the rejection of uncertainties can be used to bring the robot always close to high-confidence regions. Moreover, the GP online model update is used to aggregate the corrections received from the user to reshape the learned attractor and the stiffness field. This ensures that the proper force is executed in the correct direction for instance when cleaning a table.
To extend the learning of a skill to the whole robot pose and gripper, Chapter 5 studies how to address this with GP and with the least amount of demonstrations and corrections. Moreover, the experiments focus on teaching human-like skills to robots by exploiting the possibility of giving incremental corrections. In particular, novice users, are asked to perform the picking task of objects in one fluid motion by teaching the complete pose and gripper behavior. The execution of the skill without any supervision is usually too slow or knocks the object down before closing the gripper. Nevertheless, after providing feedback, novice users were able to incrementally shape the robot’s velocity to perform the picking at non-zero velocity, without knocking the object and correcting for any delay in gripper dynamics.
However, learning skills only relying on the current robot’s Cartesian position can be a limitation since it cannot encode skills that entail overlapping, e.g. when approaching a goal and then moving back on the same trajectory. This motivates Chapter 6 which formulates a new trajectory encoding to teach single or bimanual manipulation skills while being safe around humans with constrained velocity and force actuation. The user study also investigates the effectiveness of giving kinesthetic corrections, i.e. by simply touching the robot, and validating this in teaching bimanual skills. Teaching two manipulators at the same time or correcting them using teleoperation devices can become overwhelming. Hence, the method explores adjusting movements interactively through kinesthetic perturbations rather than re-teaching skills entirely from scratch due to imprecise attempts.
Despite the successful applications of the proposed methods in single and bimanual motion skills, during task learning, the robot must not only master the motor aspect but also be attentive to the context, such as the object’s location or shape. This motivates Chapter 7, which emphasizes the generalization of acquired motor skills across various contexts. The proposed approach hinges on GP theory to acquire a non-linear transformation map from the demonstrated task space to the execution space while preserving and propagating uncertainties. Through experiments involving tasks such as pick-and-place operations, dressing human arms, and cleaning surfaces, it is demonstrated how the robot can generalize the execution by transforming the attractor, orientation, and stiffness policy to numerous new scenario configurations even with just a single demonstration of the skill.
In Chapter 8, the concept of task parametrization and uncertainty awareness is expanded to over-parameterizing the context, such as by tracking more objects than required. The proposed algorithm would prompt user attention when encountering ambiguity, like when multiple detected objects could be the goal of the skill. Decision ambiguity can be resolved by various feedback modalities, such as pushing the robot, moving it, or providing reward/punishment. A user study also highlighted the preference of novice users for not giving conventional kinesthetic demonstrations but only intervening when necessary.@en

Performing bimanual tasks with dual robotic setups can drastically increase the impact on industrial and daily life applications. However, performing a bimanual task brings many challenges, such as synchronization and coordination of the single-arm policies. This article proposes the safe, interactive movement primitives learning (SIMPLe) algorithm, to teach and correct single or dual arm impedance policies directly from human kinesthetic demonstrations. Moreover, it proposes a novel graph encoding of the policy based on Gaussian process regression where the single-arm motion is guaranteed to converge close to the trajectory and then toward the demonstrated goal. Regulation of the robot stiffness according to the epistemic uncertainty of the policy allows for easily reshaping the motion with human feedback and/or adapting to external perturbations. We tested the SIMPLe algorithm on a real dual-arm setup where the teacher gave separate single-arm demonstrations and then successfully synchronized them only using kinesthetic feedback or where the original bimanual demonstration was locally reshaped to pick a box at a different height.

@en

Developing physically assistive robots capable of dressing assistance has the potential to significantly improve the lives of the elderly and disabled population. However, most robotics dressing strategies considered a single robot only, which greatly limited the performance of the dressing assistance. In fact, healthcare professionals perform the task bimanually. Inspired by them, we propose a bimanual cooperative scheme for robotic dressing assistance. In the scheme, an interactive robot joins hands with the human thus supporting/guiding the human in the dressing process while the dressing robot performs the dressing task. We identify a key feature: the elbow angle that affects the dressing action and propose an optimal strategy for the interactive robot using the feature. A dressing coordinate based on the posture of the arm is defined to better encode the dressing policy. We validate the interactive dressing scheme with extensive experiments and also an ablation study.

@en

A central challenge in Learning from Demonstration is to generate representations that are adaptable and can generalize to unseen situations. This work proposes to learn such a representation without using task-specific heuristics within the context of multi-reference frame skill learning by superimposing local skills in the global frame. Local policies are first learned by fitting the relative skills with respect to each frame using Gaussian Processes (GPs). Then, another GP, which determines the relevance of each frame for every time step, is trained in a self-supervised manner from a different batch of demonstrations. The uncertainty quantification capability of GPs is exploited to stabilize the local policies and to train the frame relevance in a fully Bayesian way. We validate the method through a dataset of multi-frame tasks generated in simulation and on real-world experiments with a robotic manipulation pick-and-place re-shelving task.We evaluate the performance of our method with two metrics: how close the generated trajectories get to each of the task goals and the deviation between these trajectories and test expert trajectories. According to both of these metrics, the proposed method consistently outperforms the state-of-the-art baseline, Task-Parameterised Gaussian Mixture Model (TPGMM).

@en

Adaptation to external and internal changes is of major importance for robotic systems in uncertain environments. Here, we present a novel multisensory active inference (AIF) torque controller for industrial arms that shows how prediction can be used to resolve adaptation. Our controller, inspired by the predictive brain hypothesis, improves the capabilities of current AIF approaches by incorporating learning and multimodal integration of low- and high-dimensional sensor inputs (e.g., raw images) while simplifying the architecture. We performed a systematic evaluation of our model on a 7DoF Franka Emika Panda robot arm by comparing its behavior with previous AIF baselines and classic controllers, analyzing both qualitatively and quantitatively adaptation capabilities and control accuracy. The results showed improved control accuracy in goal-directed reaching with high noise rejection due to multimodal filtering, and adaptability to dynamical inertial changes, elasticity constraints, and human disturbances without the need to relearn the model or parameter retuning.

@en

This paper studies the tuning process of controllers for fully actuated manipulators. To this end, we propose a methodology to design the desired damping matrix—alternatively, the derivative gain of a PD controller—of the closed-loop system such that n second-order systems can approximate its behavior with a prescribed damping coefficient, where n denotes the degrees of freedom of the system. The proposed approach is based on the linearization of the closed-loop system around the desired configuration and is suitable for different control approaches, such as PD control plus gravity compensation, impedance control, and passivity-based control. Furthermore, we extensively analyze simulations and experimental results in a cobot.

@en

This work investigates how the intricate task of a continuous pick & place (P&P) motion may be learned from humans based on demonstrations and corrections. Due to the complexity of the task, these demonstrations are often slow and even slightly flawed, particularly at moments when multiple aspects (i.e., end-effector movement, orientation, and gripper width) have to be demonstrated at once. Rather than training a person to give better demonstrations, non-expert users are provided with the ability to interactively modify the dynamics of their initial demonstration through teleoperated corrective feedback. This in turn allows them to teach motions outside of their own physical capabilities. In the end, the goal is to obtain a faster but reliable execution of the task. The presented framework learns the desired movement dynamics based on the current Cartesian position with Gaussian Processes (GPs), resulting in a reactive, time-invariant policy. Using GPs also allows online interactive corrections and active disturbance rejection through epistemic uncertainty minimization. The experimental evaluation of the framework is carried out on a Franka-Emika Panda. Tests were performed to determine i) the framework's effectiveness in successfully learning how to quickly pick & place an object, ii) ease of policy correction to environmental changes (i.e., different object sizes and mass), and iii) the framework's usability for non-expert users.

@en

In order to make the coexistence between humans and robots a reality, we must understand how they may cooperate more effectively. Modern robots, empowered with reliable controls and advanced machine learning reasoning can face this challenge. In this article, we presented a Disagreement- Aware Variable Impedance (DAVI) Controller, where the robot stiffness is regulated as a function of the perceived disagreement with the human cooperator. We tested the algorithm on a 7 DoF Franka Emika Panda robot performing the learning of a pick&place task with continuous adaptation of the goal location and the via-points with human interactive corrections, triggered by our proposed approach. A validation study was conducted with 5 users in order to understand the reliability of the method.@en

Teaching a robot how to navigate in a new environment only from the sensor input in an end-to-end fashion is still an open challenge with much attention from industry and academia. This paper proposes an algorithm with the name 'Learning Interactively to Resolve Ambiguity' (LIRA) that tackles the problem of sensor policy fusion extending state- of-the-art methods by employing ambiguity awareness in the decision-making and solving it using active and interactive querying of the human expert. LIRA, in fact, employs Gaussian Processes for the estimation of the policy's confidence and investigates the ambiguity due to the disagreement between the single sensor policies on the desired action to take. LIRA aims to make the teaching of new policies easier, learning from human demonstrations and correction.The experiments show that LIRA can be used for learning a sensor-fused policy from scratch or also leveraging the knowledge of existing single sensor policies. The experiments focus on the estimation of the human interventions required for teaching a successful navigation policy.

@en

Teaching robots how to apply forces according to our preferences is still an open challenge that has to be tackled from multiple engineering perspectives. This paper studies how to learn variable impedance policies where both the Cartesian stiffness and the attractor can be learned from human demonstrations and corrections with a user-friendly interface. The presented framework, named ILoSA, uses Gaussian Processes for policy learning, identifying regions of uncertainty and allowing interactive corrections, stiffness modulation and active disturbance rejection. The experimental evaluation of the framework is carried out on a Franka-Emika Panda in four separate cases with unique force interaction properties: 1) pulling a plug wherein a sudden force discontinuity occurs upon successful removal of the plug, 2) pushing a box where a sustained force is required to keep the robot in motion, 3) wiping a whiteboard in which the force is applied perpendicular to the direction of movement, and 4) inserting a plug to verify the usability for precision-critical tasks in an experimental validation performed with non-expert users.@en

In Learning from Demonstrations, ambiguities can lead to bad generalization of the learned policy. This paper proposes a framework called Learning Interactively to Resolve Ambiguity (LIRA), that recognizes ambiguous situations, in which more than one action have similar probabilities, avoids a random action selection, and uses the human feedback for solving them. The aim is to improve the user experience, the learning performance and safety. LIRA is tested in the selection of the right goal of Movement Primitives (MP) out of a candidate list if multiple contradictory generalizations of the demonstration(s) are possible. The framework is validated on different pick and place operations on a Emika-Franka Robot. A user study showed a significant reduction on the task load of the user, compared to a system that does not allow interactive resolution of ambiguities.

@en

Current ongoing industry revolution demands more flexible products, including robots in household environments and medium-scale factories. Such robots should be able to adapt to new conditions and environments and be programmed with ease. As an example, let us suppose that there are robot manipulators working on an industrial production line and that they need to perform a new task. If these robots were hard coded, it could take days to adapt them to the new settings, which would stop production at the factory. Robots that non-expert humans could easily program would speed up the process considerably.@en