The speed of a robot demonstration is often set by the teleoperator rather than by the task itself, causing the same motion to appear at widely varying paces across demonstrations. Consequently, policies trained on such data inherit operator‑specific timing alongside task behavior. To address this, we learn a speed prior directly from large‑scale demonstration data.
Given a task instruction, a motion path, and images along that path, the speed prior predicts a distribution over execution speed along the path. Selecting a specific quantile yields a consistent clock that can retime each demonstration. Unlike existing methods that must fit an acceleration signal for every new task—requiring additional fitting, annotation, or predefined rates—a single trained speed prior covers all tasks in the corpus.
We train the speed prior on both robot teleoperation data and handheld demonstrations, where a human arm (instead of a robot arm) moves the gripper. After training, we use the prior to retime robot demonstrations to the handheld pace before fine‑tuning a vision‑language‑action (VLA) policy. During execution, the policy is paired with a real‑time optimal‑control layer that tracks predicted action chunks under robot constraints using the predicted pace.
On 120 RoboLab tasks, policies fine‑tuned with our learned clocks achieve higher success rates and exhibit up to half the jerk compared with policies fine‑tuned on the original timing or uniformly accelerated to the same speed. On a real robot, the policy stacks cups in 3.50 s, 1.72× faster than the default policy’s 6.02 s, succeeding in 29 of 30 trials.
Videos and additional material are available at https://mllab.snu.ac.kr/projects/learned_speed_prior.
Review