Installation

You need:

• a Linux x86-64 workstation with an NVIDIA GPU, CUDA 12, ffmpeg and about 25 GB of free disk;
• a Franka FR3 or Panda with a Robotiq 2F-85 gripper, and its polymetis NUC;
• a wrist ZED camera, 1 third-person ZED camera and the ZED SDK;
• pipx or uv, and a Gemini API key;
• for human phases, a VR headset (Meta Quest). TANDEM builds the teleop driver's environment itself (Teleoperation); the NUC runs DROID's server (Robot setup).

Every command below runs on the workstation unless it says otherwise.

Install

pipx install git+https://github.com/SamratSahoo/TANDEM.git
# or, with uv:
uv tool install git+https://github.com/SamratSahoo/TANDEM.git

Initialize

Before you start, install the ZED SDK and plug in the cameras. Then run:

tandem init

It walks through the setup, in this order:

1. The robot: the NUC's address and the arm type.
2. TiPToP's runtime, which it builds (5–20 minutes).
3. The perception servers TiPToP calls, M2T2 and FoundationStereo, which it builds too (Perception servers).
4. The cameras. It lists the ZED cameras it finds and suggests a serial for each role (wrist, external, second external). Press Enter to accept a suggestion or type another serial. Then it asks which camera perception reads.
5. Your Gemini key.
6. The paper's five tasks, added as profiles.
7. Teleoperation, if you want it.

You can run it again at any time; it skips steps that are already done. tandem init --repair asks every question again.

Check the setup

tandem doctor   # every check, and what to do about each; --no-hardware skips the robot, camera and server probes

Common problems are in Troubleshooting.