ARBot: A High-Fidelity Robotic Manipulator Teleoperation Framework for Human-Centered Augmented Reality Evaluation
Worcester Polytechnic Institute
- Full Repository Installation
- Install Procedure for Particular Submodules
- Dataset
- Demo Videos
- Citation
- License
To download the repositories with all the dependencies, use the following command:
git clone --recurse-submodules https://github.com/cake-lab/ARBot.gitUse the table below as a reference to build and run your code:
| Sub-Module Name | Content | Step-by-Step Implementation Procedure |
|---|---|---|
| AR-PiPER | ROS2 Package to simulate and control the PiPER Arm | AR-PiPER |
| AR-OMX | ROS2 Package to simulate and control the Robotis Open Manipulator X Arm | AR-OMX |
| AR-CV | Vision-based Teleoperation system for the CV+IMU control setup | AR-CV |
| ARPose | Android Studio Files for the ARPose Mobile Application | ARPose |
The Full Dataset (132 User Trajectories) can be found at Dataset.
These are a few demo videos for your reference. We will have more demo videos for you shortly.
- CV+IMU Interface
- CV+IMU Demo
- ARPose Application Interface
- ARPose Application Demo;
- Synthetic/Autopilot Test Demo
- Repeatability Demo
If you use ARBot in your research, please cite:
@inproceedings{10.1145/3793853.3799807,
author = {Chhajed, Harsh and Guo, Tian},
title = {ARBot: A High-Fidelity Robotic Manipulator Teleoperation Framework for Human-Centered Augmented Reality Evaluation},
year = {2026},
isbn = {9798400724817},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3793853.3799807},
doi = {10.1145/3793853.3799807},
abstract = {Validating Augmented Reality (AR) tracking and interaction models requires precise, repeatable ground-truth motion. However, human users cannot reliably perform consistent motion due to biome-chanical variability. Robotic manipulators are promising to act as human motion proxies if they can mimic human movements. In this work, we design and implement ARBot, a real-time teleoperation platform that can effectively capture natural human motion and accurately replay the movements via robotic manipulators. ARBot includes two capture models: stable wrist motion capture via a custom CV and IMU pipeline, and natural 6-DOF control via a mobile application. We design a proactively-safe QP controller to ensure smooth, jitter-free execution of the robotic manipulator, enabling it to function as a high-fidelity record and replay physical proxy. We open-source ARBot and release a benchmark dataset of 132 human and synthetic trajectories captured using ARBot to support controllable and scalable AR evaluation.},
booktitle = {Proceedings of the ACM Multimedia Systems Conference 2026},
pages = {409–415},
numpages = {7},
keywords = {Augmented Reality, Robot Teleoperation, Human-Robot Interaction},
location = {
},
series = {MMSys '26}
}This project is licensed under the terms of the LICENSE file in this repository.