An augmented-reality-assisted immersive system for robotic arms teleoperation

Robotic teleoperation allows humans to remotely operate to perform several tasks in different fields, from social to high-risk situations. In this paper, we propose a novel immersive system which exploits augmented reality (AR) to provide effective feedback to the operator, simplifying the task and optimizing the performance during teleoperation. The AR cues are conveyed to the operator through a head-mounted display (HMD) equipped with hand tracking and gesture recognition system to control the remote robotic arms for teleoperation, and a head orientation system to control the orientation of a remote RGB-D camera. This setup allows to reduce the bandwidth required for video streaming (both 2D and point cloud) and to improve the quality of the remote interaction in real-world tasks.