A Shared Control Interface for Online Teleoperated Teaching of Combined High- and Low-level Skills

Abstract

We propose a novel shared control interface that enables teleoperated teaching of both high-level decision-making skills and low-level impedance modulation skills using a single haptic device. In the proposed method, high-level teaching is achieved by repurposing the haptic device to remotely modify Behaviour Trees (BTs), allowing human operators to guide decision-making. Repurposing of the haptic device is achieved by exploiting its degrees of freedom for different functionalities. Low-level skill teaching involves an impedance command interface, that is used to command endpoint stiffness by manipulating a 3D virtual stiffness ellipsoid with the haptic device. Both teaching modes are connected: a newly demonstrated low-level skill appears in the BT at a user-specified index. Control is shared between the human and the autonomous system on a high- and low-level. At the higher level, the human can change the BT online, while ongoing execution of the low-level actions within behavior tree remains uninterrupted. During low-level teaching, shared control is implemented between the robotic motion skill and human-demonstrated stiffness. To provide a proof-of-concept and demonstrate the main features of the proposed interface, we performed several experiments in a teleoperation setup operating a remote shelf-stocker robot in a supermarket environment. A predefined BT encodes high-level decisions for a pick-and-place task. The impedance command interface is evaluated in a “peg-in-hole”-like task of placing a product on a cluttered shelf. Ultimately, the proposed interface can facilitate teleoperation-based Learning from Demonstration for the transfer of both high- and low-level skills in an integrated manner.