Trajectory‑level GRPO is a common approach for training multi‑step GUI agents, yet repeatedly rolling out full trajectories from the task root is inefficient: task‑level rewards give only coarse supervision, making it hard to pinpoint which intermediate decisions need improvement, and already‑mastered prefixes are re‑executed unnecessarily. To address this, we introduce a two‑stage adaptive training framework that jointly considers where to learn and how far to roll out. First, we build reusable intermediate checkpoints from successful expert trajectories and select informative pivots using action disagreement, which directly measures uncertainty from sampled GUI actions.
Stage I performs one‑step reinforcement learning at these pivots to refine uncertain local decisions. Stage II progressively expands policy responsibility according to task progress, conducting Local, Suffix, and Root training respectively. Recoverable checkpoints allow training to resume from intermediate states without replaying preceding trajectories.
Across MobileGym and CUA‑Gym‑Hub, our method reduces environment interactions by 56%–61% compared to Vanilla GRPO while improving task performance, demonstrating that jointly allocating training across states and rollout ranges can substantially boost the efficiency of GUI‑agent RL.
Review