Modern robot imitation learning increasingly relies on generative policies based on diffusion or flow‑matching models, which generate actions by transforming samples drawn from a prior distribution. A central question is whether the choice of prior matters. Prior work has shown that, when training from scratch, replacing the standard Gaussian prior $\mathcal{N}(0, I)$ with a non‑Gaussian prior that is closer to the target distribution $p_{\text{target}}$ can substantially boost performance.
This paper asks whether the same gains transfer to fine‑tuning pretrained Large Behavior Models (LBM 1.0, $\pi_{0.5}$, and GR00T~N1.5). One might expect even larger improvements, especially under limited fine‑tuning data. We conduct over 100K simulation rollouts across 40+ tasks on two simulators, plus 1,250 hardware rollouts on five bimanual manipulation tasks. The results are surprising: except at very low data fractions, non‑Gaussian priors that are demonstrably closer to the target yield statistically indistinguishable or even worse fine‑tuning performance compared to the standard Gaussian prior.
Diagnostic analyses reveal that fine‑tuned imitation policies converge to nearly identical action predictions across priors, even though the fine‑tuned encoder embeddings diverge substantially from both the pretrained embeddings and each other. A learning‑rate ablation confirms that encoder training dominates fine‑tuning performance, outweighing the effect of prior choice by a large margin.
We conclude that, for fine‑tuning large pretrained behavior models, the prior distribution is not a decisive factor. Future work should investigate scenarios where learned priors might still matter, such as extreme low‑data regimes, cross‑domain transfer, or when imposing specific constraints on the action space.
Review