DATAFARM: Distribution-Aligned Task and Motion Planning for Fine-Tuning Vision-Language-Action Models

Samrat Sahoo1, Yixuan Huang2, Tom Silver2
1Stanford University · 2Princeton University
Paper under review
arXiv Code

Abstract

Collecting high-quality robot data remains a fundamental challenge for training robot foundation models. Task and motion planning (TAMP) offers a scalable way to generate demonstrations, but our experiments show that raw TAMP trajectories provide surprisingly little benefit when used to fine-tune pretrained vision-language-action (VLA) models, despite successfully solving the target tasks. We hypothesize that this failure arises from a behavioral distribution mismatch between planner-generated trajectories and the data used to pretrain the VLA.

To address this mismatch, we introduce DATAFARM: Distribution-Aligned Task And motion planning for Fine-tuning A Robot foundation Model, an approach that incorporates the pretraining distribution directly into TAMP trajectory generation. DATAFARM aligns generated trajectories with the pretraining data in robot joint configurations, motion style, and temporal execution profiles. We evaluate DATAFARM on three tabletop manipulation tasks that TAMP can perform and a cloth-folding task beyond the capability of TAMP. DATAFARM achieves an average success rate of 56.7%, substantially outperforming raw TAMP (8.3%) while approaching human teleoperation (61.7%). On Deformable Object Manipulation, which is outside the fine-tuning distribution, the fine-tuned model retains 85% success, compared with 90% for the pretrained model. These results show that aligning planner-generated demonstrations with the pretraining distribution can make TAMP an effective source of data for VLA fine-tuning.

Given a pretrained VLA and a new task, TAMP is cheap but off-distribution, teleoperation is on-distribution but expensive, and distribution-aligned TAMP (DATAFARM) is cheap and on-distribution. End-effector speed profiles show DATAFARM resembling teleoperation.
TAMP data is scalable but mismatched with the VLA’s pretraining data; teleoperation matches it but needs substantial human effort. DATAFARM aligns TAMP trajectories with the pretraining distribution.

Method

DATAFARM learns models of reference behavior from pretraining data and uses them to guide the planner’s configurations, paths, and timing.

DATAFARM overview: a Gaussian mixture density over robot configurations and a trajectory encoder with a corresponding latent distribution are learned from pretraining demonstrations; these provide alignment costs for TAMP endpoint selection, path optimization, and retiming; the resulting trajectories are executed on the robot to collect demonstrations used to fine-tune the pretrained VLA.
Overview of DATAFARM. A Gaussian mixture over joint configurations and a trajectory encoder, both learned from pretraining demonstrations, provide alignment costs for endpoint selection, path optimization, and retiming.
1 · Joint space

Endpoint Selection

For each end-effector pose, rank the IK solutions by configuration cost, favoring arm postures with higher reference density.

2 · Motion style

Path Optimization

Add the trajectory cost to the planner’s objective and optimize the waypoints by gradient through the frozen encoder. This changes how the arm moves between endpoints; feasibility is checked before acceptance.

3 · Timing

Retiming

Keep each path’s geometry fixed and optimize how quickly the robot traverses it, matching reference speeds around gripper events. Motion limits are checked before acceptance.

1 · Endpoint selection
A density over robot joint configurations with several peaks; an arrow runs from the highest peak to the configuration selected on the trajectory. Caption: We choose the IK solution that minimizes the negative log-likelihood of its joint configuration under a Gaussian mixture fit to the pretraining data.
2 · Path optimization
3 · Retiming

Tasks

Each checkpoint is fine-tuned on 20 successful demonstrations and evaluated for 20 trials.

Initial and final scenes for Geometric Constraints, Multi-Step Reasoning, Semantic Reasoning, and Deformable Object Manipulation.
  • Geometric Constraints targetPlace three toys on a plate without collisions; earlier placements limit the free space.
  • Multi-Step Reasoning targetRemove an obstruction from a tray, then place two toys in it.
  • Semantic Reasoning targetPlace three objects into two bowls by semantic category.
  • Deformable Object Manipulation out-of-distributionFold a cloth. The planner cannot do this, so it is excluded from fine-tuning and tests retention.

Raw TAMP vs. DATAFARM demonstrations

Both are successful planner executions; DATAFARM’s are aligned with DROID in joint configurations, motion style, and timing.

Raw TAMP
Geometric Constraints
Multi-Step Reasoning
Semantic Reasoning
DATAFARM
Geometric Constraints
Multi-Step Reasoning
Semantic Reasoning

DATAFARM runs without human control; humans only set up and reset scenes.

Results

Raw TAMP demonstrations solve the tasks yet barely help fine-tuning: a task-valid trajectory is not necessarily good fine-tuning data.

Success rate and task progress for task-specific fine-tuning
Show full results table
ApproachGeometric ConstraintsMulti-Step ReasoningSemantic Reasoning
Succ.Prog.OOD Succ.OOD Prog.Succ.Prog.OOD Succ.OOD Prog.Succ.Prog.OOD Succ.OOD Prog.
π0.5-DROID0%18%90%95%0%13%90%95%5%50%90%95%
Raw TAMP5%35%85%93%5%37%70%85%15%58%75%88%
DATAFARM w/o style10%39%––0%31%––35%54%––
DATAFARM w/o timing0%19%––0%18%––0%29%––
DATAFARM w/o joint space35%73%––15%53%––20%66%––
DATAFARM65%84%80%90%35%81%85%88%70%89%90%93%
Human teleop (oracle)50%93%95%98%45%90%95%98%90%98%95%98%

OOD: retention on Deformable Object Manipulation, which is excluded from fine-tuning. Teleoperation is an oracle only in its access to human demonstrations.

56.7% average success. Up from 8.3% for raw TAMP and 1.7% for π0.5-DROID, with gains on every target task.
Close to teleoperation, without teleoperating. 56.7% vs. 61.7% with human demonstrations; humans only set up and reset scenes.

Policy rollouts after fine-tuning on DATAFARM

Each fine-tuned policy on its target task.

Target tasks
Geometric Constraints
Multi-Step Reasoning
Semantic Reasoning
Deformable Object Manipulation out-of-distribution
Before fine-tuning (π0.5-DROID)
After fine-tuning on DATAFARM

No fine-tuning dataset includes this task, yet the policies retain 85% average success.

Contribution of each alignment component

Removing one component at a time lowers average success from 56.7% to 23.3% without joint space, 15.0% without style, and 0% without timing.

Ablations
Retention of pretrained capabilities

Cloth folding: 90% before fine-tuning; 80%, 85%, and 90% after fine-tuning on each target task.

Effect of demonstration budget

Geometric Constraints; 0 is π0.5-DROID without fine-tuning. Success reaches 75% at 40 demonstrations and 85% at 80.

Alignment with DROID

Samples projected onto the first two principal components of (a) joint configurations and (b) trajectory embeddings.

PCA density contours. Joint space: 74% of DATAFARM samples inside the DROID contour vs. 66% raw TAMP and 60% without joint-space alignment. Latent space: 96% DATAFARM vs. 0% raw TAMP, 89% without style, 19% without timing.

Joint space: 74% of DATAFARM samples lie inside the DROID contour, vs. 66% for raw TAMP and 60% without joint-space alignment.

Trajectory latent space: 96% for DATAFARM, vs. 0% for raw TAMP, 89% without style, and 19% without timing.

Overlap alone does not ensure success: the style ablation reaches 89% overlap but only 15.0% success.

Autonomous Data Collection

Using TAMP, the robot can alternate between collecting a demonstration and resetting the scene.

Nine rollouts and eight resets in 14 minutes.

BibTeX

@article{datafarm2026,
  title   = {DATAFARM: Distribution-Aligned Task and Motion Planning
             for Fine-Tuning Vision-Language-Action Models},
  author  = {Sahoo, Samrat and Huang, Yixuan and Silver, Tom},
  year    = {2026},
  note    = {Under review}
}