LeWAM is an end‑to‑end Joint‑embedding Predictive Architecture (JEPA) designed for control tasks, learning a compact latent world model and action‑conditioned dynamics within a single network. The key contributions are:
- A joint training objective that simultaneously minimizes next‑latent‑state prediction loss and action‑flow matching loss, ensuring the latent space captures both environment evolution and action sensitivity.
- An isotropic Gaussian constraint on the latent state, $z \sim \mathcal{N}(0, I)$, which standardizes the latent distribution so the same model can be used for direct policy execution and for fast planning by evaluating sampled actions through the learned dynamics.
On visual planning benchmarks, LeWAM raises success rates from 28.6% to 89.7% on contact‑rich tasks and from 10.9% to 46.2% on long‑horizon tasks, while achieving a 32.3× speed‑up in planning. In behavior‑cloning experiments on simulators and real robots, LeWAM remains competitive with established baselines. Interpretability studies show that the learned latent representations focus on control‑relevant features and that the action distribution effectively guides test‑time planning and control.
Review