The general approach to generate collision free motion in a constraint environment is to use path planners, which demand a known environment and potentially fail otherwise. Learning from Demonstration (LfD) can be used instead to teach the robot unknown parts of the environment, such as a goal deviation or an unforeseen obstacle. The general approach is to train a model offline and expect it to perform well afterwards. Problems arise however when the model is trained insufficiently or unknown variations have occurred in the environment, which demand for refinement of the model. In online learning the operator is allowed to refine a predicted trajectory during execution time, where the state-of-the-art methods focus on kinaesthetic teaching. The contribution of this research is the development of a teleoperated online learningmmethod, where the operator can make refinements by moving a haptic stylus (Phantom Omni) in the desired direction. By doing this, a force is felt proportional to the magnitude of refinement. After creating such a refined trajectory, it is used to update a condition dependent probabilistic trajectory model. The proof of concept was shown on a 2D example and on a simulated robot that shows that we can adapt an initial model when unknown variations occur and that the method is able to deal with different object positions and initial end effector poses. To show if other people can use the method, a human factors experiment is performed, comparing the developed method against three other methods on how much time it takes to successfully adapt a model (refinement time) and on the perceived workload. Two different parameters are varied, which are the teaching device (stylus or keyboard) and the learning mechanism (online or offline). The expectation was that both online with stylus has the lowest refinement time and workload, but the results show that only online has a significant improvement over offline methods (p = 7.94 × 10^−12 and p = 0.000512 respectively). This is explained by the fact that only small corrections have to be made and in a maximum of three degrees of freedom (DoFs). No significant difference was found between keyboard and stylus (p = 0.755 and p = 0.302 respectively). An explanation for this is that this is task, person and implementation dependent. The recommendations are to evaluate the proof of concept on the real robot and to extend the method with orientation refinement, such that more complex tasks can also be dealtwith. We hypothesize that with these tasks the combination of online with stylus does perform the best.

Haptic assistance (HA) has been shown to be useful in a wide variety of applications by providing the benefits of automation while keeping the human in the loop. In recent years, providing additional visual feedback has been found to provide benefits to HA interfaces by complementing their advantages and mitigating their weaknesses. However, the implementation and understanding of this interaction are still quite limited. In this study, we provide new insights into this interaction by extending these findings to the HA teleoperation of an industrial robot, in which the HA is comprised of a set of Virtual Fixtures (VFs). Two methods of providing additional visual feedback are implemented for the use case of dross removal and are evaluated in a human factors experiment. The first method is the implementation of a set of visual cues, designed to complement the HA and force feedback. The second method is the use of a head-mounted display (HMD), instead of a desktop monitor, providing additional depth information and an increased sense of immersion. Both methods proved to be beneficial, but only in certain aspects of the operation. The visual cues were found to significantly improve safety in terms of peak collision force, whereas the HMD significantly improves the performance. Additional analysis suggests the use of an HMD causes improvements in the manipulability of the interface. Furthermore, improved scores in the van der Laan questionnaires and the user preference indicate an increased user acceptance due to the implementation of either of the methods. This study provides additional insight into the importance of visual feedback for HA and provides two methods to take advantage of its potential benefits in the teleoperation of an industrial robot.