This thesis explores the challenge of achieving haptic bilateral teleoperation across long distances, focusing on enhancing Model Mediated Teleoperation (MMT) to support dynamic environments under high network latency. Instead of striving for perfect model alignment, we acknowledge the inevitable mismatch between the remote environment and its model at the operator. We suggest a set of design considerations and a complementary framework to make operator intent a key factor in MMT solution development. By developing a system where a user can control a remote robot arm to draw on a moving whiteboard using a novel position-altering scheme that attempts to prioritize operator intent, we apply the framework to minimize the impact of network latency on user experience. Our findings reveal that the implemented position-altering scheme significantly reduces the effects of network delays on drawing accuracy, and a user study confirms that force feedback delays are more critical to user satisfaction than visual feedback delays. The results suggest that satisfactory teleopera-tion performance can be achieved with network latency up to 50 ms, challenging the conventional 10 ms requirement. This research provides design considerations, a comprehensive framework with a position-altering policy, and an open-source software package for advancing haptic teleop-eration technologies.

Haptic bilateral teleoperation technology aims to transmit the sensation of touch over the internet, enabling remote work for all. However, it faces significant challenges, including stringent latency requirements (sub 1 ms) for the Tactile Internet and the need for advanced models in Model Mediated Teleoperation. This work explored the feasibility of haptic bilateral teleoperation over long distances through a new framework that leverages live video and predictive haptic feedback. This framework was implemented on a real experimental setup where users draw with a haptic device controlling a remote robotic arm on a moving whiteboard. A user study was conducted on this setup which confirmed that the method is sound and revealed that force feedback latency requirements are significantly more stringent than those of visual feedback. In this application users did not perceive the application as more difficult at 50 ms network latency (110 ms visual latency) with user experience only degrading at 75 ms added network latency. These findings prove that through the Demonstrate Imitate Adjust framework, long-distance haptic bilateral teleoperation is achievable.

Bilateral teleoperation with force feedback aims to transmit human expertise over long distances by transferring the sensation of physical contact. One of the primary challenges in achieving this goal is the ultra low latency requirement. Tactile internet and model-mediated teleoperation are promising research areas addressing the latency constraint. In model-mediated teleoperation, it is crucial that the remote environment parameters are tracked with minimal latency to update the local model. This work investigates the potential of designing a cost-effective object tracking solution that performs comparably to state-of-the-art systems while maintaining low tracking latency. A method is proposed to streamline the design process by leveraging assumptions about the tracking conditions. An object-tracking algorithm that can effectively fuse high-frequency noisy inertial data (>1 kHz) with delayed, low-frequency (30 Hz) but accurate camera data has been designed. A practical system was constructed using cost-efficient components, and a dataset was collected for testing and characterization. The system demonstrated the ability to produce object pose estimates at 1 ms intervals. In experiments along a single axis, the system achieved a mean positional estimation error of 0.1 cm and a mean orientation estimation error of 0.5◦. The algorithm also successfully corrected for camera latencies of up to 350 ms while maintaining accurate estimates. These results demonstrate the possibility of achieving a low latency, accurate object tracking system while keeping component and computation costs reasonable.

Tactile Internet opens many new possibilities as it solves one of the critical problems in the modern-day Internet, namely latency. Remote operations are possible that are deemed too dangerous with current technologies. Interacting directly with the remote environment without being physically present will be a life-changing experience. Before building the Tactile Internet infrastructure, continued research is necessary for haptic technology and networking. A modern Tactile Internet testbed is needed for researchers to conduct studies. The testbed uses Unity with Bullet Physics as an external physics engine and includes network possibilities and basic 3D tracking. Latency analysis shows a relation between the physics engine execution time and the scene's complexity. The testbed can handle many objects in the environment if continuous interactions among objects are limited. Force feedback analysis shows the expected behavior for the virtual spring implementation.

Tactile Internet (TI) allows kinesthetic interactions with a remote environment and haptic sensory feedback over a network, essentially adding a new sensory dimension to the internet. TI has a wide range of applications such as enabling remote work for professions that require human hands and tactile sensory input like car repairs or medical operations. Most TI applications have ultra low latency requirements that limit TI to short distances because the speed of light poses a constraint on how fast data can be transmitted over a distance. This thesis attempts to relax the delay requirements for enabling TI over long distances. This is achieved by locally simulating the physical interactions and tactile properties of the environment to provide instant haptic feedback regardless of network conditions.

Model Mediated Teleoperation (MMT) creates a local feedback loop using a model and lowers the network delay requirements of the system at large. We propose utilizing a physics engine with haptic functionalities as the local model, to create MMT based TI applications that can cater for a variety of complex scenarios. We implement a multi-purpose haptic framework attached to Unity that facilitates creating various TI testbeds and applications, and use it to analyze and benchmark our MMT design. We show that haptic feedback from stationary objects is completely unaffected by network delay with our MMT implementation, and narrow down the challenge to simulating dynamic objects. We demonstrate that the model inevitably diverges from the environment it simulates when movable objects are involved, and needs frequent corrections.

Humans interact more and more at a distance these days. Where currently, this interaction is mainly visual and auditory, Tactile Internet (TI) also allows for remote kinesthetic interactions. With the devel- opment of TI, a paradigm shift is on the horizon where the current internet evolves from content delivery to a skill-set delivery network. Imagine the possibilities when specialists in whatever field can, in real- time, apply their skills everywhere in the world from within their living room. To make this a reality, TI applications have to be able to provide feedback to the user with ULL. The speed of light, however, constrains the extent to which TI can be used at large distances. This thesis hence takes a closer look at the problems that arise when long-distance TI is to be used and tries to tackle them. These problems are tackled by building a system based on Model-Mediated Teleoperation (MMT), where lo- cal simulations of the environments are used. A local feedback loop is instantiated through these local simulations, which lowers the system’s dependence on the network performance. The former control loop, including the network, now becomes a corrective loop to prevent mismatches. This thesis focuses on the part where the local simulation is corrected by the real-life data received over the network. The question is how real-time dynamic environments can be captured and kept track of for use in MMT. We identify accurate tracking of objects in the environment as the critical building block enabling MMT with dynamic objects. Consequently, the first 6 Degrees of Freedom (DoF) tracking application designed for use in TI applications is created and evaluated. Through this research, the first steps are set towards a full-blown MMT system that can capture the remote environment successfully.

Tactile internet promises to enable humans to manipulate remote physical environments. The human interacts with the remote environment by manipulating a haptic device. Recently, research effort into tactile internet and haptic devices has seen a significant increase. However replicating one aspect of executing an action, feeling textures, is still in its infancy. Current devices that can portray texture are severely limited, which also interferes with the ability to research the topic. In this thesis, we propose a novel device that can realistically portray textures to the fingertip, and respond to displacement in real-time. We call this a Haptic Texture Device (HTD). The design features custom actuators, with low latency < 0.6ms and at least 50mN per mm2 of pushing force. The actuators are placed in a grid with a 2.0mmpitch at the fingertip. TheHaptic TextureDevice (HTD) can be moved over a 2D surface. Movement of the HTD results in real-time updates to the portrayed texture to reflect the displacement. The end-to-end delay between displacement and update of the texture was measured to be 17 ms. However, we demonstrated that the theoretical limit of the end-to-end delay could bewell below1 ms, by making use of higher quality displacement sensor. Readily available heightmaps are used as a source of texture data. Because a method for characterizing the performance hitherto did not exist, a novel benchmark was developed that incorporates realistic textures and human participants. The results of our benchmarking yielded an average score of 61%. Since the device is the first of its kind, it cannot be compared to similar devices yet, but it can be used as a starting point on which future HTDs can improve.

Platooning is an application of autonomous driving in which the vehicles travel linearly following each other. The benefits of such driving are many-fold, primarily it increases the traffic throughput and also reduces fuel consumption as the result of minimized drag experienced by the vehicles. The state-of-the-art technologies to achieve platooning involve vehicle-to-vehicle communication for receiving all the events instantaneously. However, wireless channels being prone to communication losses pose risks to the platoon operation. Therefore, it is important to study how platoons operate under communication losses. Platoons being vehicular ad-hoc networks, the information dissemination is governed by Time Division Multiple Access policies known as Information Flow Topology (IFT). Platooning being sensitive to communication losses, during an emergency braking, the requirements are even more stringent as it could lead to rear-end collisions within the vehicles.

All the performance evaluation are done on a newly designed testbed which is both cost-effective and customizable. Several kinds of IFTs have been developed in recent years. We consider four of the predominant ones and evaluate their performance during emergency braking with temporary communication loss. Since the decrease in inter-vehicle distances is the main reason for vehicle collisions during platoon, we use it as a key performance indicator when evaluating the performance. To improve the reliability of communication in the platoon, we propose to adopt an existing methodology in wireless networking and combine it with the IFTs in platooning. We additionally show the improvement in the collision probability with this method. We conclude our work with limitations and further improvements.

Tactile Internet (TI) is, the next frontier of connectivity development wherenot only information but interaction can be exchanged digitally. Much effort in the field of TI has been spent towards developing the necessary technologies needed to transmit interactions over the internet in a naturally perceivedway. However not much has been done on improving the evaluation of these TI technologies, that is, Mean Opinion Score score or Peak Signal to Noise Ratio measures for audio-video applications. Existing measures and measuring methods only give coarse-grained indications of performance and lackthe depth for good comparison. By using Dynamic Time Warping as starting point we propose our new measures Effective Time & Value -Offset that fundamentally changes the way to evaluate performance. We verify our measures by showing it performs significantly better compared to existing work by using data from a new testbed designed to allow testing in stringent network conditions and to be usable for many futureworks.Our contributions will push forward TI development allowing both incremental and novel developments to be tested more easily and with improved verification.