NeFut Logo NeFut
Admin Login

[CS.AI] Making Latent Evolution Explicit: Operator-Structured Transitions for World Action Models

Published at: 2026-09-21 22:00 Last updated: 2026-09-22 02:29
#algorithm #Machine Learning #Artificial Intelligence

World Action Models (WAMs) enhance robot policies by forecasting how task‑relevant scene states evolve under interaction. Recent WAMs increasingly perform this prediction in latent spaces, avoiding full appearance‑level generation while preserving control‑relevant information. Most existing approaches rely on Transformer‑based predictors whose inductive bias focuses on token interaction rather than temporal evolution.\

This work treats transition realization as an architectural choice separate from representation learning and policy coupling, and introduces the Latent Evolution Operator Network (LEON). LEON models latent evolution in a learned observable space via two complementary pathways: context‑modulated operator propagation and additive forcing. Grounded in the controlled Koopman generator perspective, the dynamics can be written as\ $$\dot{z}=K(z)z + f(z, c)$$\ where $K(z)$ is a shared evolution operator and $f$ is an additive term modulated by context $c$. This design forces context‑dependent transition variations to orbit a common operator structure while retaining a separate channel for non‑linear changes.\

Controlled dynamical‑system experiments confirm that operator propagation encodes a strong evolution‑specific inductive bias, while the forcing term complements it by handling residual dynamics. Across two WAM formulations that integrate latent prediction into the policy in different ways, LEON consistently improves closed‑loop performance and robustness, and remains effective even when the entire transition module is swapped out. These results establish transition realization as a consequential architectural decision in latent WAMs.\

Review: By making latent evolution explicit through a shared operator, LEON balances global dynamical consistency with context‑specific flexibility, offering a promising direction for more adaptable robot control in complex environments.

Original Source: https://arxiv.org/abs/2608.27259

[h] Back to Home