In recent years, providing additional visual feedback about the interaction forces has been found to offer benefits to haptic-assisted teleoperation. However, there is limited insight into the effects of the design of force feedback-related visual cues and the type of visual display on the performance of teleoperation of robotic arms executing industrial tasks. In this study, we provide new insights into this interaction by extending these findings to the haptic assistance teleoperation of a simulated robotic arm in a virtual environment, in which the haptic assistance is comprised of a set of virtual fixtures. We design a novel method for providing visual cues about the interaction forces to complement the haptic assistance and augment visual feedback in virtual reality with a head-mounted display. We evaluate the visual cues method and head-mounted display method through human factors experiments in a teleoperated dross removal use case. The results show that both methods are beneficial for task performance, each of them having stronger points in different aspects of the operation. The visual cues method was found to significantly improve safety in terms of peak collision force, whereas the head-mounted display additionally improves the performance significantly. Furthermore, positive scores of the subjective analysis indicate an increased user acceptance of both methods. This work provides a new study on the importance of visual feedback related to (interaction) forces and spatial information for haptic assistance and provides two methods to take advantage of its potential benefits in the teleoperation of robotic arms.@en

Robotic teleoperation is used in various applications, including the nuclear industry, where the experience and intelligence of a human operator are necessary for making complex decisions that are beyond the autonomy of robots. Human-robot interfaces that help strengthen an operators situational awareness without inducing excessive cognitive load are crucial to the success of teleoperation. This paper presents a novel visual interface that allows operators to simultaneously control a 6-DoF camera platform and a robotic manipulator whilst experiencing the remote environment through a virtual reality head-mounted display (HMD). The proposed system, Orbital Head-Mounted Display (OHMD), utilizes head rotation tracking to command camera movement in azimuth and elevation directions around a fixation point located at a robot's end-effector. A human factor study was conducted to compare the interface acceptance, perceived workload, and task performance of OHMD with a conventional interface utilizing multiple fixed cameras (Array) and a standard head-mounted display implementation (HMD). Results show that both the OHMD and HMD interfaces significantly improve task performance, reduce perceived workload and increase interface acceptance compared to the Array interface. Participants reported they preferred OHMD due to the increased assistance and freedom in viewpoint selection. Whilst OHMD excelled in usefulness, the standard HMD interface allowed operators to perform robotic welding tasks significantly faster.@en

Unlike many traditional stiff position-controlled robots, new collaborative robots interact with humans and operate in an environment that is often unpredictable and unknown. For safe and effective executions of manipulation tasks within such an environment, the robot requires to modulate its compliance. Therefore, the human operator must have a system that enables an intuitive demonstration of compliance skills to the robot. Ideally, this should also be possible through teleoperation in order to have the ability to demonstrate skills at a distance, such as in remote home care applications or any other scenario where the skilled operators are not physically present all the time. Existing state-of-the-art methods for remote demonstration of impedance skills only enable limited modulation of the stiffness matrix, or they are too complex and cumbersome for practical applications. This research tries to overcome these limitations and proposes a teleoperated stiffness commanding method that enables a complete modulation of the stiffness matrix in 3 degrees of freedom. The method uses the same haptic device hardware as used for controlling the robot manipulator motion, hence it does not require extra specialised equipment for stiffness commands. By wiggling the endpoint of the haptic device, the stiffness is commanded to the robot and also fed back to the operator through haptic and visual feedback. To evaluate the performance and acceptance of the system, we performed a user study where the participants had to demonstrate various interaction behaviour to the remote robot. The results show how varying system parameters (i.e., degrees of freedom, orientation, and size of the stiffness commands) influence the performance of the system and user acceptance.@en