Most text‑to‑SQL benchmarks provide the target database directly, yet in a large catalog the choice of sources is part of the problem. An agent must locate every database required for a request, determine how their results fit together, and execute the resulting workflow. DSABench is introduced to evaluate these three capabilities jointly.
The benchmark comprises 815 human‑reviewed tasks drawn from Spider, BIRD, and LiveSQLBench. Among them, 618 tasks need contributions from several databases, and 105 allow more than one plausible interpretation of the required sources. Each task includes an executable contract that defines source participation, cross‑source dependencies, and source‑local execution constraints. RouteEnv lets agents inspect public catalog evidence without revealing gold annotations, enabling fair routing decisions.
In a routing‑only study across seven model backends, the highest RA‑Set score reaches 60.55%, while the best RA‑Graph score drops to 34.66% when coordination requirements are added. The gap widens further when execution is required. Across four backends, the median non‑privileged Route+Exec‑Graph success ranges from 6.69% to 7.98% without clarification and from 11.44% to 13.25% with a single clarification; the best clarified configuration attains 16.99%. On ambiguous tasks, clarification improves exact branch‑source recovery by 39.8–43.2 percentage points.
These findings illustrate that merely retrieving plausible databases or producing correct local SQL is insufficient: an agent must carry the right information throughout the entire multi‑database workflow to achieve end‑to‑end success.
Review