Adding a human executor

A human executor carries out a human phase and records it as one leg. It only decides how the phase is done: the plan decides which phase comes next, and the camera check decides whether it worked.

TANDEM ships one executor, teleop. A package can add others, such as a learned policy. The code is in src/tandem/executors/.

Choosing one

tandem executors list           # each executor: ready, needs setup (and what), or broken
tandem executors use mypolicy   # the active profile now uses it; -p PROFILE for another

use sets the profile's hitl.human_executor. It warns if the executor isn't ready, and refuses an unknown or broken one (HTTP API too).

• A profile that names a missing executor can't collect until use fixes it. It can still be browsed, exported and edited.
• With phase planning on, a session looks the executor up before warm-up, so a broken plugin fails early. tandem doctor's human executor row reports it.
• While recording, an executor that isn't ready leaves a human phase only a (give up). d also works if hitl.allow_unrecorded_human_phase is true.

Registering one

A minimal executor that runs a policy, and the factory that registers it:

# my_package/executor.py
from tandem.executors import ExecutorFactory, HumanPhaseResult

class PolicyExecutor:
    name = "mypolicy"
    segment_source = "policy"
    display_name = "My policy"
    summary = "Runs a learned policy."

    def __init__(self, ctx):
        self.ctx, self._killed = ctx, False

    def run(self, request, leg, *, save_root, should_stop):
        try:
            # yours: run the policy, recording under save_root/"eval"; poll should_stop() and self._killed
            leg_dir, n_frames, stopped = self._run_policy(request, leg, save_root, should_stop)
            status = "aborted" if self._killed else "ended_by_operator" if stopped else "done"
            return HumanPhaseResult(status, n_frames=n_frames, leg_dir=leg_dir)
        finally:
            self._killed = False

    def kill(self):
        self._killed = True

FACTORY = ExecutorFactory(
    create=PolicyExecutor,
    display_name=PolicyExecutor.display_name,
    summary=PolicyExecutor.summary,
    segment_source=PolicyExecutor.segment_source,
    requirements=("a policy checkpoint",),
    check=lambda settings: [],
)

Register it with an entry point in pyproject.toml:

[project.entry-points."tandem.human_executors"]
mypolicy = "my_package.executor:FACTORY"

Or call tandem.register_human_executor("mypolicy", FACTORY) (or "my_package.executor:FACTORY"). Keep the module light: listing executors imports it.

• check(settings) returns unmet requirements in words. An empty list means ready. Keep it cheap and start nothing.
• A class can be the factory. It needs segment_source, display_name, summary and requirements class attributes, and optionally validate_options. It can't have check.
• Names follow the planner rule. Reusing a name needs replace=True. Two installed packages claiming one name is an error.

ExecutorContext

create receives this once per session.

If the factory has validate_options(options), it runs when the profile loads. It returns plain data, which is saved to the profile's file, or raises TandemError or ValueError naming the bad key.

The protocol

Implement tandem.executors.HumanExecutor. Every member is required except close().

The arguments to run:

If the arm can't be freed, raise tandem.executors.CustodyError, and the session ends. teleop does this when its driver survives a kill.

HumanPhaseRequest

HumanPhaseResult

HumanPhaseResult(status, n_frames=0, leg_dir=None, leg_dirs=())

n_frames counts every frame the hand-off wrote. leg_dir is the last recording's directory (or None), and leg_dirs lists every recording in order (empty means just leg_dir).

While recording, a result with n_frames=0 is refused and the phase is asked again, without spending a retry. hitl.allow_unrecorded_human_phase: true accepts it instead. Giving up with a runs no executor, so it is not aborted.

Custody

Each human leg runs in four steps:

1. backend.release_hardware() frees the robot and cameras. A failure only warns.
2. executor.run(...) runs. It is skipped as aborted if a forced stop came during step 1.
3. The leg is counted. It is labeled and merged even if step 4 fails.
4. backend.reacquire_hardware() takes the robot back, unless run raised CustodyError. A failure ends the session.

Any other exception from run fails the trial at human_policy, and the session goes on.

The teleop executor

TeleopExecutor runs the DROID teleop driver, one process per leg. It needs:

• the teleop runtime, from tandem executors install teleop (setup). It holds the workstation side of DROID's TANDEM branch and oculus_reader, in their own environment;
• teleop.enabled (keys), which the install turns on;
• a VR headset and controller (Meta Quest).

TANDEM sets DROID_NUC_IP and TIPTOP_*_CAMERA_ID for the driver from the rig, which DROID's droid/misc/parameters.py reads. To run a DROID checkout and environment of your own instead, set both teleop.droid_dir and teleop.python; the driver then runs there, and the runtime is ignored.

tandem executors list checks teleop.enabled and the runtime (or both overrides), but not the device. A missing headset shows up as a driver error when the leg starts.

If the driver can't start, the leg stays open and shows the problem until you return control. One hand-off can make several recordings, and all of them are in leg_dirs.