Vision‑Language models are increasingly deployed for multimodal QA, yet isolating their ability to reconstruct latent spatial structure from a single image remains challenging. Existing benchmarks typically conflate perception, OCR, domain knowledge, linguistic priors and reasoning, making results hard to interpret.
We introduce StateSight, a procedurally generated benchmark comprising three task families: opposite‑face reasoning on a cube‑net, counting occluded cube towers, and 4‑neighbor connected‑component counting. Each family offers 300 single‑image prompts with deterministic oracle labels and exact‑match scoring.
In public evaluation, OpenAI GPT‑5.5 (API identifier gpt-5.5) achieved accuracies of 59.3%, 33.3% and 28.3% across the three tasks, while Claude Sonnet 5 recorded 53.3%, 18.7% and 7.3%. All direct runs produced zero format errors. A human baseline with 30 participants on 60 items yielded mean accuracies of 80.8%, 68.8% and 64.3%, surpassing both models on every task.
Visible‑derivation analysis revealed recurring errors in image‑state reconstruction and reasoning procedures. To facilitate deeper study, we also release StateSight‑Steps, a companion dataset containing 900 interleaved image‑text examples and 3,600 deterministic intermediate visual states. The findings demonstrate that format‑valid responses can mask failures to recover the spatial structure required for verifiable visual inference.
Blogger's Review: StateSight cleanly isolates spatial reasoning capabilities of vision‑language models, offering a clear diagnostic tool for future improvements. Extending the benchmark with richer geometric variations could further probe models' general spatial understanding.