CARLA-GS is a modular corner-case synthesis pipeline aimed at evaluating safety in autonomous driving, particularly focusing on rare and safety-critical interactions. Existing knowledge and model-based methods typically isolate scene or trajectory components, while diffusion-based approaches, though attempting end-to-end generation, still struggle with spatiotemporal consistency and physical realism.
The design of CARLA-GS decouples visual representation, semantic reasoning, and physics-based execution while maintaining tight coupling between modules. Starting from real driving data, we reconstruct an editable Gaussian scene with additional geometry-consistent constraints. A multi-agent LLM performs scene-level reasoning to identify risky interactions and generate intent-level waypoint trajectories, while low-level motion control is delegated to CARLA, where a PID controller ensures kinematic and dynamic feasibility. The simulated vehicle states are then re-projected into the Gaussian scene for ego-centric rendering.
Experimental results demonstrate that our framework enables controllable corner-case generation, producing photorealistic, spatiotemporally consistent videos aligned with semantic intent and physically feasible motion. This study shows both quantitative and qualitative results on the Waymo Open Dataset, proving its effectiveness.
Blogger's Review: The innovation of CARLA-GS lies in its modular design that successfully decouples different processing stages, enhancing the safety and reliability of autonomous driving systems. By utilizing LLM for higher-level reasoning, the system can generate more intelligent and safer driving scenarios, which is worthy of further research and promotion.