Haptic guidance is a promising method for assisting an operator in solving robotic remote operation tasks. It can be implemented through different methods, such as virtual fixtures, where a predefined trajectory is used to generate guidance forces, or interactive guidance, where sensor measurements are used to assist the operator in real-time. During the last years, the use of learning from demonstration (LfD) has been proposed to perform interactive guidance based on simple tasks that are usually composed of a single stage. However, it would be desirable to improve this approach to solve complex tasks composed of several stages or gestures. This paper extends the LfD approach for object telemanipulation where the task to be solved is divided into a set of gestures that need to be detected. Thus, each gesture is previously trained and encoded within a Gaussian mixture model using LfD, and stored in a gesture library. During telemanipulation, depending on the sensory information, the gesture that is being carried out is recognized using the same LfD trained model for haptic guidance. The method was experimentally verified in a teleoperated peg-in-hole insertion task. A KUKA LWR4+ lightweight robot was remotely controlled with a Sigma.7 haptic device with LfD-based shared control. Finally, a comparison was carried out to evaluate the performance of Gaussian mixture models with a well-established gesture recognition method, continuous hidden Markov models, for the same task. Results show that the Gaussian mixture models (GMM)-based method slightly improves the success rate, with lower training and recognition processing times.

@en

Series elastic actuators (SEAs) are interesting for usage in harsh environments as they are more robust than rigid actuators. This paper shows how SEAs can be used in teleoperation to increase output velocity in dynamic tasks. A first experiment is presented that tested human ability to achieve higher hammerhead velocities with a flexible hammer than with a rigid hammer, and to evaluate the influence of the resonance frequency. In this experiment, 13 participants executed a hammering task in direct manipulation using flexible hammers in four conditions with resonance frequencies of 3.0 Hz to 9.9 Hz and one condition with a rigid hammer. Then, a second experiment is presented that tested the ability of 32 participants to reproduce the findings of the first experiment in teleoperated manipulation with different feedback conditions: with visual and force feedback, without visual feedback, without force feedback, and with a communication delay of 40 ms. The results indicate that humans can exploit the mechanical resonance of a flexible system to at least double the output velocity without combined force and vision feedback. This is an unexpected result, allowing the design of simpler and more robust teleoperators for dynamic tasks.@en

In haptic shared control systems (HSC), a fixed strength of guidance force equates to a fixed level of control authority, which can be insufficient for complex tasks. An adaptable control authority based on operator input can allow the HSC system to better assist the operator under varied conditions. In this paper, we experimentally investigate an adaptable authority HSC system that provides the operator with a direct way to adjust the control authority based on applied grip force. This system can serve as an intuitive override function in case of HSC system malfunction. In a position tracking task, we explore two opposite approaches to adapt the control authority: increasing versus decreasing guidance strength with operator grip. These approaches were compared with unassisted control and two levels of fixed-level haptic guidance. Results show that the grip-adaptable approach allowed the operators to increase performance over unassisted control and over a weak guidance. At the same time, the approach substantially reduced the operator physical control effort required to cope with HSC system disturbances. Predictions based on the formalized model of the complete human-in-the-loop system corresponded to the experimental results, implying that such validated formalization can be used for model-based design of guidance systems.@en

Future human spaceflight exploration missions to the Moon and beyond are hypothesised to benefit from human-robotic integrated operations. The European Space Agency focuses on preparing these operations, following the objective stated in the Global Exploration Roadmap of the International Space Exploration Coordination Group. Currently, human-robotic operations aim at technology development and demonstration, yet essential questions that remain unanswered are: How can human performance be measured, and which metrics are used? This contribution aims at answering this by preparing a pilot phase focusing on human performance assessment for subjects controlling a rover. A recent study by Hosseini (2016) for ESA identified essential knowledge gaps that must be filled to assess astronaut performance for tele-operations, which is of great importance for future missions since they affect mission planning, task allocation and even tool selection. In this regard, this study proposes follow-on research in which a tele-operations experiment is conducted by driving a rover, in order to evaluate human performance in tele-operations. In the previous study, two space-to-ground and multiple ground-to-ground tele-operations experiments were analysed in which subjects controlled a rover. Data analysis studied the command time and execution time of the assigned tasks, i.e. the time it takes for the human to give the command to the rover and the time it takes for the rover to execute its tasks, respectively. Results showed that the main challenge for performance assessment is the lack of recorded parameters. The logged data is limited only to time values and success/failure results and it does not specify performance variations. Furthermore, the study concludes that many different sub-tasks were performed in a limited amount of time, resulting in scarce data per sub-task and limiting the statistical significance. Follow-on research is proposed that aims at solving for the two above mentioned issues. Firstly, the study introduces parameters, which are used to assess the performance of pilots regarding neuro-ergonomics and human factors. Secondly, an experiment is set up and is tested for its rigidity in a protocol rehearsal, prior to performing the experiment with a relatively large group of participants. It is hypothesised that this approach has the potential not only to increase the qualitative assessment of the performance, but also to increase the quantitative results essential for preparing crew training and future missions.@en

Teleoperation – performing tasks remotely by controlling a robot – permits the execution of many important tasks that would otherwise be infeasible for people to carry out directly. Nuclear accident recovery, deep water operations, and remote satellite servicing are just three examples. Remote task execution principally offers two extremes for control of the teleoperated robot: direct tele manipulation, which provides flexible task execution, but requires continuous operator attention, and automation, which lacks flexibility but offers superior performance in predictable and repetitive tasks (where the human assumes a supervisory role). This dissertation explores a third option, termed hap- tic shared control, which lies in-between these two extremes, and in which the control forces exerted by the human operator are continuously merged with ‘guidance’ forces generated by the automation. In a haptic shared control system, the operators continually contribute to the task execution, keeping their skills and situational awareness. It is common practice to design the haptic shared control systems heuristically, by iteratively adjusting them to the satisfaction of the system designer, primarily based on human-in- the-loop experiments. In this dissertation, we aim to improve this design and evaluation process. Our goal is to follow a system-theoretic approach and formalize the design procedures of haptic shared control systems applied to teleoperation. Such a formalization should provide designers of future HSC systems with a better understanding and more control over the design process, with the ultimate goal of making the HSC systems safer, easier and more intuitive to use, and overall to perform better. The research goal of this dissertation has been divided into three parts.@en

Haptic guidance is a promising way to support unmanned aerial vehicle (UAV) operators, but the design of haptic guidance forces is often heuristic. This paper describes the design and experimental validation of a systematic neuromuscular analysis-based tuning procedure for haptic guidance, here applied to haptic collision avoidance system for UAV teleoperation. This tuning procedure is hypothesized to reduce operator workload as compared with current heuristic tuning methods. The proposed procedure takes into consideration the estimated mechanical response of the neuromuscular system (NMS) to haptic cues. A “relax-task” setting of the NMS, for which reflexive and muscular activation is minimal, is chosen as the design point for tuning the haptic support, as this setting is expected to yield minimal physical workload. The paper first presents a neuromuscular identification experiment, performed to estimate the “relax task” admittance of an operator's arm. The averaged admittance of a group of subjects (n=10) was then used for tuning the haptic shared controller, which was subsequently evaluated in its ability to support different operators (n=12) in a simulated unmanned aerial vehicle surveillance task. Results show that our novel tuning procedure indeed reduces operator workload and also improves situation awareness compared with haptic settings that ignore the NMS. In fact, it is shown that overtuning, which frequently occurs for these heuristically tuned systems, leads to even lower user acceptance scores than interfaces without any haptic support.@en

In this paper, we propose a closed-loop force sensor based nested admittance/impedance control strategy to actively estimate and minimize the effects of geometric misalignment that naturally occur during assembly tasks with compliant robots. The method allows the robot to be used with a stiff impedance control setting, which is beneficial for free air motion performance, yet allows to adjust for large misalignment errors between parts that need be assembled.

@en

This paper introduces a new Learning from Demonstration (LfD)-based method that makes usage of robot effector forces and torques recorded during expert demonstrations, to generate force-based haptic guidance reference trajectories on-line, that are intended to be used during haptic shared control for additional operator 'guidance'. Derived haptic guidance trajectories are superimposed to master-device inputs and feedback forces within a bilateral control experiment, to assist an operator by the guidance during peg-in-hole insertion. We show that 96 peg-in-hole expert demonstrations were sufficient to obtain a good model of the task, which was used on-line to generate haptic guidance trajectories in real-time with a 1kHz sampling rate.

@en

On July 28th 2014, 23:47 UTC, the European Space Agency launched the Haptics-1 Kit to the International Space Station (ISS) on its last Automated Transfer Vehicle ATV-5. The Kit reached the station two weeks later, marking the first haptic master device to enter the ISS. The first force-feedback and human perceptual motor performance tests started to take place on December 30th 2014, and are the first of their kind in the history of spaceflight. Three astronauts participated in the Haptics-1 experiment until November 2015, allowing the investigation of the effects of microgravity on various psycho-motor performance metrics related with the usage of haptic feedback. Experiments are conducted following full adaptation to the space environment (after 3 months in space). This paper introduces the Haptics-1 experiment and associated hardware. Detailed experimental results are reported from a first stiffness just noticeable difference (JND) experimental study in space, carried out on the ISS and pre-flight on ground with 3 astronauts. The first findings from the experiment show no major alterations in-flight, when compared to on-ground data, if the manipulandum is secured in flight against a sufficiently stiff reference structure.

@en

This paper introduces a new Learning from Demonstration (LfD)-based method that makes usage of robot effector forces and torques recorded during expert demonstrations, to generate force-based haptic guidance reference trajectories on-line, that are intended to be used during haptic shared control for additional operator 'guidance'. Derived haptic guidance trajectories are superimposed to master-device inputs and feedback forces within a bilateral control experiment, to assist an operator by the guidance during peg-in-hole insertion. We show that 96 peg-in-hole expert demonstrations were sufficient to obtain a good model of the task, which was used on-line to generate haptic guidance trajectories in real-time with a 1kHz sampling rate.

@en

Learning from Demonstration has been successfully used in robotics for trajectory generation. However, this methodology has not been used to solve generic tasks by haptic guidance in teleoperation yet. Therefore, the aim of this paper is to solve the peg-in-hole insertion task using Learning from Demonstration, guiding the operator during the execution of this task in haptic teleoperation.@en

In a typical space teleoperation task, mismatches between the viewing direction of the operator and the direction of their required control input are often unavoidable. To execute these tasks, the operator is then required to perform mental rotations. Recent studies have shown that the task performance can thereby significantly decrease. In this paper, for the first time, the influence of mental rotations on task performance is studied if hap tic feedback is provided to the operator. A human factors experiment is conducted which analyses the influence of two different hap tic feedback control methods via various visual missmatch angles. The rotation is thereby set to the extreme cases of 0. and 180.To clearly analyze the effects. The first hap tic feedback method consists of direct, scaled force and torque feedback to the operator as measured by a force/torque sensor at the slave robot. The second method consists of hap tic shared control which provides artificially generated guidance forces to the operator. It is shown that mental rotations decrease teleoperation performance despite the addition of direct force feedback. In contrast, hap tic shared control provides lower increase in the operator mental workload and also less between-operator variability of errors made due to the mental rotations.

@en