Long‑horizon GUI agents need visual history to keep task states, yet storing every screenshot inflates visual token usage and inference cost. Pure textual summaries often discard crucial state cues, harming long‑term reliability. ECHOES leverages privileged information available during training to guide textual history compression and strengthen long‑horizon reasoning.
ECHOES adopts a Compression‑Think‑Action (CTA) format: first compress the current visual input into a textual history state, then reason over that state, and finally emit an action. The same model, when fed multi‑image privileged inputs, acts as a teacher that provides dense token‑level on‑policy distillation for student‑generated CTA rollouts.
To balance privileged distillation with task‑reward optimization, ECHOES introduces a Perception‑Memory Fidelity (PMF) Gap‑guided OPD‑RL framework that dynamically weights the two objectives and periodically updates the teacher for continual improvement.
We also present ECHOESBMK, a fine‑grained diagnostic benchmark derived from real GUI agent failure trajectories, jointly evaluating visual perception and cross‑modal state retention.
Experiments on four GUI benchmarks and ECHOESBMK show substantial gains in task success and memory fidelity. Without retaining historical screenshots at deployment, ECHOES matches the performance of screenshot‑based methods while offering a favorable performance‑inference‑cost trade‑off, and it exhibits positive zero‑shot transfer to unseen visual planning tasks.
Review