NeFut Logo NeFut
中 Admin Login

[CS.AI] Correct Answers, Closed Gates: SolveBind for GUI Agents at the Verification Barrier

Published at: 2026-10-26 22:00 Last updated: 2026-10-27 00:43
#AI #Machine Learning #LLM

Websites and applications often deploy CAPTCHAs and anti‑automation mechanisms that rely on behavioral signals—such as interaction trajectories and timing—rather than just final outcomes to separate humans from bots. In these protected GUI environments, an agent must not only solve the verification task but also turn the solver’s output into interactions that the system accepts, adapting execution on‑the‑fly under time constraints.\ \ We introduce SolveBind, a GUI‑agent framework that couples hierarchical solving with runtime supervision. The verification problem is decomposed into five inter‑dependent capabilities—Perceive, Ground, Solve, Act, Verify—forming a solving tree. A runtime supervisor grounds action commitments in tool‑derived evidence, revises them as new evidence arrives, and mediates execution within explicit recovery budgets.\ \ Across nine CAPTCHA families and nine backbone models, SolveBind achieves the highest success rate on eight backbones, outperforming the strongest baseline by 13.3–34.5 percentage points. It also attains the top success rate on all six AndroidWorld backbones. In a paired analysis with GPT‑5.5, MobileAgent produces correct answers but fails dynamic verification on 13 of 90 tasks; SolveBind succeeds on seven of those.\ \ Automatic submission experiments show that removing the binding step drops success from 82.22% to 71.56% while substantially increasing latency and token usage. Recovery caps reduce execution cost with a modest success trade‑off: removing them raises success to 83.56% but inflates latency and token consumption to 3.7× and 2.4× their full‑supervision values, respectively.\ \ These findings support evidence‑grounded execution control and highlight a success‑cost trade‑off in recovery strategies and heuristic submission rules.\ \ Review

Original Source: https://arxiv.org/abs/2610.32223

[h] Back to Home