NeFut Logo NeFut
中 Admin Login

[CS.AI] When MLLMs Miss What Humans Can Read: Diagnosing and Learning to Read Squint Text

Published at: 2026-10-13 22:00 Last updated: 2026-10-14 02:30
#AI #Machine Learning #LLM

Multimodal large language models (MLLMs) can miss text embedded in an image's coarse structure—text that humans can read by squinting—yet report no text present. We call this phenomenon Squint Text and treat it as a recognition deficiency, motivated by cases from a production moderation pipeline. To study it we built SquintBench, containing 1076 moderation images from real deployments, and SquintCtrl, a controlled synthetic suite for mechanism diagnosis.

Frozen‑weight analysis shows that hidden text leaves a coarse signal while glyph identity is weakly represented; providing the correct spatial regions makes identity substantially more accessible, indicating spatial selection as a major bottleneck. Suppressing high‑frequency content also improves reading under a fixed visual‑token budget, and the token count itself may contribute.

These insights lead to Squint OPSD: a frozen teacher receives a downscaled view and the ground‑truth transcription as privileged context, and its token‑level distributions supervise a student along on‑policy trajectories. At inference the student reads the original image alone. On SquintBench the method reaches 88.61 ANLS, far above the unadapted backbone's 26.50, the best general‑purpose model on the original view's 24.44, and the best off‑the‑shelf model under a downscaled view's 68.60, substantially narrowing the gap between visual evidence and generated read‑out without auxiliary views.

Review: The work highlights spatial selection and frequency information as key factors for recognizing tiny text and offers an efficient fine‑tuning approach that does not require extra views at inference.

Original Source: https://arxiv.org/abs/2610.13465

[h] Back to Home