IGG is an interactive GUI grounding benchmark designed to evaluate the ability to make a target visible under visibility constraints and then click it. Each task requires first revealing the target through interaction before performing the click. The benchmark comprises 601 tasks across 16 controllable GUI environments and is divided into seven visibility‑constraint sub‑types. The action space is compact, limited to scrolling, moving, dragging, waiting, and clicking. Human participants achieve 94.6% success, GPT‑5.5 reaches 79.4%, and the strongest open‑source VLM attains only 31.6%. To pinpoint the gap, two interventions are tested: informing Qwen3‑VL which interaction to perform raises accuracy from 30.8% to 33.9%, while externally performing the interaction for the model lifts accuracy to 81.9%. This shows the bottleneck lies in executing the reveal interaction rather than recognizing the target once exposed. Additional experiments demonstrate that visibility recovery can be learned and consistently improved across model families.
Review