Robotic Characterization of Markerless Hand-Tracking on Meta Quest Pro and Quest 3 Virtual Reality Headsets.

Markerless hand-tracking has become increasingly common on commercially available virtual and mixed reality headsets to improve the naturalness of interaction and immersivity of virtual environments. However, there has been limited examination of the performance of markerless hand-tracking on commercial head-mounted displays. Here, we propose an evaluation methodology that leverages a robotic manipulator to measure the positional accuracy, jitter, and latency of such systems and provides a standardized characterization framework of markerless hand-tracking.