VLA‑Scope is a two‑stage framework that predicts failure during out‑of‑distribution rollouts of vision‑language‑action models. The first stage uses pooled image and language embeddings to detect OOD inputs and assign them to shift categories. For inputs flagged as OOD, the second stage combines the predicted category, action‑prefix features, and execution‑progress features in a logistic regression model shared across categories, updating failure risk as execution proceeds.
We evaluate the approach on OpenVLA across ten LIBERO‑Spatial tasks using leave‑one‑group‑out cross‑validation. OOD detection attains a ROC‑AUC of 0.9454 and shift classification reaches 91% accuracy. When applied to all 1,400 OOD rollouts, the failure predictor, independent of the OOD gate, achieves a ROC‑AUC of 0.8497 after 60 actions, outperforming the 0.7906 baseline without progress features and surpassing ActProbe and SAFE‑MLP baselines. These results suggest that fusing action features with temporally aggregated execution step representations improves failure prediction under input shifts.
Review