BABA-Bench is a benchmark composed of 47 interactive worlds built from de‑identified real patient‑advocacy cases reviewed by experts. In each world the agent must uncover hidden information and coordinate insurers, clinics, suppliers, pharmacies, and community organizations. The engine checks every action in real time for patient permission, evidential support, and prohibited operations. We evaluated 14 frontier models with three runs per world, totaling 1,974 runs. Many agents reached the correct care outcome but followed invalid procedures. Among the 921 runs that achieved all required outcomes, 254 (28%) violated the necessary process. Claude Opus 5 achieved all outcomes in 81.6% of its runs, yet only 52.5% passed all checks. When we supplied a verified route to two models, their pass rates rose from 29.8% to 91.5% and from 44.0% to 97.2%, respectively. The main challenge lies in finding a compliant route rather than executing actions.
Review: BABA-Bench highlights a critical compliance gap in current LLM agents for healthcare, suggesting that future work should integrate path planning with procedural adherence.