The Planner-Operator-Reflector (POR) framework is widely adopted in desktop GUI agents, maintaining objective alignment through modular collaboration of a planner, an operator, and a reflector.
Desktop GUIs, however, are often large and information‑dense, with state changes that are subtle and scattered. This pushes most of the workload onto the reflector, which must compare pre‑ and post‑action screens, while the planner and operator reason over a single state.
Existing reflectors collapse change detection and outcome verification into a single step, leaving evidence implicit and resulting in weakly grounded decisions.
To address this, we propose Evidence-First Reflection (EFR), a two‑stage reflector that first extracts action‑induced visual differences explicitly, then verifies the outcome.
EFR employs Set‑of‑Marks annotations to mark the action location and candidate changed regions, describes and filters changes relevant to the action, and finally makes a judgment based on the cleaned evidence.
This evidence‑first design grounds reflection directly in screen transitions, substantially reducing visual search complexity and reasoning burden.
Experiments on OSWorld‑Verified and WindowsAgentArena benchmarks show that EFR improves reflector accuracy by 7.11%, yielding average end‑to‑end task success gains of 5.94% and 4.95% respectively.
Blogger's Review: The two‑stage decoupling of evidence extraction and reasoning in EFR provides a more reliable evidence chain for GUI agents, especially in complex desktop environments, and demonstrates clear performance gains that merit broader adoption in real‑world systems.