Wild test‑time adaptation (WTTA) updates a source model online under small test batches, concurrent distribution shifts, and time‑varying class imbalance. Existing WTTA methods derive adaptation signals—such as predictive uncertainty, sample reliability, and local feature geometry—solely from the model being adapted. When the source model is unreliable under shift, these signals can amplify its own errors, creating a self‑referential loop.
We introduce MASA (Multimodal‑LLM‑Anchored Semantic Adaptation), which supplements model‑internal evidence with structured semantic descriptions from a frozen multimodal large language model (MLLM). To limit inference cost, MASA queries the MLLM only for a small set of diverse, reliability‑ranked anchors. The resulting descriptions capture the object family as well as nuisance factors like style, viewpoint, and occlusion.
MASA encodes these descriptions, propagates them to neighboring test samples, and stores the resulting visual‑semantic information in an online prototype memory. Descriptor‑aware retrieval from this memory provides an auxiliary target for lightweight adaptation of normalization‑affine parameters.
We evaluate MASA on the WTTA ImageNet‑C benchmark under limited‑batch, mixed‑domain, and imbalanced‑label‑shift settings using ResNet and ViT backbones, demonstrating its effectiveness in challenging test‑time scenarios.
Review