NeFut Logo NeFut
Admin Login

[CS.AI] ObsDriveBench: Benchmarking Multimodal Understanding in Adverse Weather

Published at: 2026-07-29 22:00 Last updated: 2026-07-30 03:24
#algorithm #AI #Machine Learning

Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions, synthetic corruptions, or single modality. Thus, it's unclear how vision-language models behave under real-world adverse weather with multi-modal inputs. We argue that a key difficulty lies in degraded environmental observability: under fog, rain, snow, and low illumination, multi-modal observations become unreliable and cross-modally inconsistent, posing challenges to scene understanding and subsequent decision-making. To address this, we introduce ObsDriveBench, a real-world multi-modal benchmark for adverse-weather autonomous driving. Our benchmark is designed with three capability dimensions: observability awareness, spatial reliability, and risk-aware decision-making, enabling fine-grained diagnosis of model behavior under degraded observations. We construct the benchmark through observability meta-annotation, scene description, and capability-oriented multiple-choice tasks over synchronized camera, LiDAR, and radar inputs, forming a benchmark with over 14k training and 13k test questions. Experiments reveal consistent performance degradation of existing vision-language models. We further introduce the ObsDrive model with normal-weather supervised fine-tuning and adverse-weather reinforcement learning, improving robustness across all three capabilities. The dataset and evaluation code will be released at ObsDriveBench.

Blogger's Review: This study addresses a significant issue in the autonomous driving field regarding effective multimodal understanding under adverse weather conditions, presenting an innovative solution. ObsDriveBench not only enriches existing benchmarks but also provides crucial directions for future research through detailed observability analysis.

Original Source: https://arxiv.org/abs/2607.23537

[h] Back to Home