ST-Bench is a benchmark created to assess whether multi‑agent systems (MAS) can outperform single agents on real scientific data‑analysis tasks. It adapts 100 data‑science problems from published Earth‑science studies in hydrology, agriculture, and wetland methane research, expanding them into 2,067 queries that are grounded in additional literature and validated by domain experts.
Using the same GPT‑5 backbone, we evaluate five recent MAS generation methods under two training protocols and compare them with single‑agent baselines. Nine out of ten MAS configurations surpass the cheapest single‑agent baseline, and the strongest configuration achieves nearly three times the composite score.
The gain is mainly due to coverage: the trained workflows produce realistic numerical metrics for a larger fraction of queries, while the quality of those metrics—conditioned on producing realistic output—is comparable to the single‑agent baseline. The top configuration requires about four times the inference time of a single agent, whereas a more economical workflow captures most of the benefit at less than twice the cost.
In summary, MAS specialization provides measurable advantages for scientific data analysis, but the benefit is conditional rather than universal.
Review