In recent years, providing additional visual feedback about the interaction forces has been found to offer benefits to haptic-assisted teleoperation. However, there is limited insight into the effects of the design of force feedback-related visual cues and the type of visual display on the performance of teleoperation of robotic arms executing industrial tasks. In this study, we provide new insights into this interaction by extending these findings to the haptic assistance teleoperation of a simulated robotic arm in a virtual environment, in which the haptic assistance is comprised of a set of virtual fixtures. We design a novel method for providing visual cues about the interaction forces to complement the haptic assistance and augment visual feedback in virtual reality with a head-mounted display. We evaluate the visual cues method and head-mounted display method through human factors experiments in a teleoperated dross removal use case. The results show that both methods are beneficial for task performance, each of them having stronger points in different aspects of the operation. The visual cues method was found to significantly improve safety in terms of peak collision force, whereas the head-mounted display additionally improves the performance significantly. Furthermore, positive scores of the subjective analysis indicate an increased user acceptance of both methods. This work provides a new study on the importance of visual feedback related to (interaction) forces and spatial information for haptic assistance and provides two methods to take advantage of its potential benefits in the teleoperation of robotic arms.@en

Daily household tasks involve manipulation in cluttered and unpredictable environments and service robots require complex skills and adaptability to perform such tasks. To this end, we developed a teleoperated online learning approach with a novel skill refinement method, where the operator can make refinements to the initially trained skill by a haptic device. After a refined trajectory is formed, it is used to update a probabilistic trajectory model conditioned to the environment state. Therefore, the initial model can be adapted when unknown variations occur and the method is able to deal with different object positions and initial robot poses. This enables human operators to remotely correct or teach complex robotic manipulation skills. Such an approach can help to alleviate shortages of caretakers in elderly care and reduce travel time between homes of different elderly to reprogram the service robots whenever they get stuck. We performed a human factors experiment on 18 participants teaching a service robot how to empty a dishwasher, which is a common daily household task performed by caregivers. We compared the developed method against three other methods. The results show that the proposed method performs better in terms of how much time it takes to successfully adapt a model and in terms of the perceived workload.

@en

Robotic teleoperation is used in various applications, including the nuclear industry, where the experience and intelligence of a human operator are necessary for making complex decisions that are beyond the autonomy of robots. Human-robot interfaces that help strengthen an operators situational awareness without inducing excessive cognitive load are crucial to the success of teleoperation. This paper presents a novel visual interface that allows operators to simultaneously control a 6-DoF camera platform and a robotic manipulator whilst experiencing the remote environment through a virtual reality head-mounted display (HMD). The proposed system, Orbital Head-Mounted Display (OHMD), utilizes head rotation tracking to command camera movement in azimuth and elevation directions around a fixation point located at a robot's end-effector. A human factor study was conducted to compare the interface acceptance, perceived workload, and task performance of OHMD with a conventional interface utilizing multiple fixed cameras (Array) and a standard head-mounted display implementation (HMD). Results show that both the OHMD and HMD interfaces significantly improve task performance, reduce perceived workload and increase interface acceptance compared to the Array interface. Participants reported they preferred OHMD due to the increased assistance and freedom in viewpoint selection. Whilst OHMD excelled in usefulness, the standard HMD interface allowed operators to perform robotic welding tasks significantly faster.@en

Unlike many traditional stiff position-controlled robots, new collaborative robots interact with humans and operate in an environment that is often unpredictable and unknown. For safe and effective executions of manipulation tasks within such an environment, the robot requires to modulate its compliance. Therefore, the human operator must have a system that enables an intuitive demonstration of compliance skills to the robot. Ideally, this should also be possible through teleoperation in order to have the ability to demonstrate skills at a distance, such as in remote home care applications or any other scenario where the skilled operators are not physically present all the time. Existing state-of-the-art methods for remote demonstration of impedance skills only enable limited modulation of the stiffness matrix, or they are too complex and cumbersome for practical applications. This research tries to overcome these limitations and proposes a teleoperated stiffness commanding method that enables a complete modulation of the stiffness matrix in 3 degrees of freedom. The method uses the same haptic device hardware as used for controlling the robot manipulator motion, hence it does not require extra specialised equipment for stiffness commands. By wiggling the endpoint of the haptic device, the stiffness is commanded to the robot and also fed back to the operator through haptic and visual feedback. To evaluate the performance and acceptance of the system, we performed a user study where the participants had to demonstrate various interaction behaviour to the remote robot. The results show how varying system parameters (i.e., degrees of freedom, orientation, and size of the stiffness commands) influence the performance of the system and user acceptance.@en

Telemanipulation techniques allow for human-in-the-loop assembly and maintenance tasks in otherwise inaccessible environments. Although it comes with limitations in achieved performance - required strict operator selection and extensive training are widely encountered - there is very little quantitative insight in the exact problems operators encounter during task execution. This paper provides a novel hierarchical task analysis approach to identify the most time-consuming subtask elements and to quantify the potential room for performance improvement during telemanipulated maintenance tasks. The approach is illustrated with a human factors case study in which 5 subjects performed six generic maintenance tasks, using a six degree of freedom master device connected to a simulated task environment. Overall it can be concluded that the proposed three phased task analysis is a useful tool to guide improvements since it is able to relate high-level problems (e.g. large variability) to behaviour on lower task-levels. For the case study, the largest potential for improvement was found for specific subtasks characterized by complex contact transitions and precise control of tool orientation, and in the reduction of variation of the task execution.@en

Certain tele-manipulation tasks require manipulation by two asymmetric slaves, for example a crane for hoisting and a dexterous robotic arm for fine manipulation. It is unclear how to best design human-in-the-loop control over two asymmetric slaves. The goal of this paper is to quantitatively compare the standard approach of two co-operating operators that each control a single subtask, to a single operator performing bimanual control over the two subtasks, and a uni-manual control approach.

@en

For the planned teleoperated maintenance of the experimental fusion plant ITER the time performance will be critical. Telemanipulated task execution is however characterised by long execution times compared to similar tasks performed hands-on. There is little quantitative research on task performance of telemanipulated maintenance available to give insight into most effective areas for improvement.In this paper a detailed analysis of real world remote maintenance at fusion plant JET is performed with the aim to: i) identify bottlenecks in task completion time and ii) quantify the room for potential improvement.Video recordings of the installation of 50 tiles executed by the three official master-slave operators were analysed. The task execution was characterised by a large variation in time performance, between but also within operators. Reduction of this variation could theoretically result in time reduction up to 41%. Recurring tasks like 'rough/fine approach' and 'retreat' covered more than 50% of the total task completion time and were identified as most promising for further improvement.The results will be the base for further research on operator assistance with augmented visual or haptic guidance.@en

In tele-operation, haptic feedback from the remote environment to the human is often limited, which has been shown to negatively influence the performance and required time of tasks. The conventional research focus is on improving the quality of the haptic feedback (transparency), which may have led to significant improvement, but is still imperfect, with many unresolved issues. The present study presents an alternative approach to improve tele-operated tasks: by offering haptic shared control in which both operator and support system apply the required forces at the input (master) device. It is hypothesized that virtual forces from well-designed shared control will improve required time and accuracy, with less control effort, and that these benefits exist for perfect transparency but even more so for imperfect transparency. In an experimental study haptic shared control was designed to aid operators (n=9) with performing a simple bolt-spanner task using a planar (2D, 3DOF) tele-operator setup. The experimental results provided evidence for the hypotheses, showing that the tested tele-manipulation task benefits from haptic shared control, for three different levels of transparency. Essentially, the presence of haptic shared control allows for a worse transparency without compromising required time, and can even improve required time during perfect transparency.

@en