NeFut Logo NeFut
Admin Login

[CS.AI] Does Splitting a Triage Decision Across Agents Hide Bias or Help Catch It?

Published at: 2026-08-10 22:00 Last updated: 2026-08-11 02:05
#LLM #Artificial Intelligence #Audit Capacity

Recent benchmarking work has shown that a single large language model (LLM), forced to make life-or-death resource-allocation decisions, exhibits measurable demographic bias. Real deployments, however, rarely use a single agent: they use pipelines, with review steps meant to catch exactly this kind of failure. We study what happens to bias when the same decision is distributed across a role-differentiated multi-agent pipeline (assessment, allocation, independent audit) instead of made and checked by one model alone. Using a synthetic disaster-triage simulator with paired cases that are clinically identical except for one demographic attribute, we run 192 episodes (2,304 resolved case pairs) on GPT-4o-mini comparing a single-agent control condition to a nine-agent pipeline under three independently varied pressure dimensions. We find no measurable difference in how often biased outcomes occur between the two conditions (6.9% vs. 6.1%, p = 0.498). We do find a large and significant effect of audit capacity on whether bias is caught: 30.0% of biased outcomes go entirely undetected, rising to 43.8% when the auditor is overloaded and falling to 18.4% when it is not. Decomposing this effect shows it is driven almost entirely by coverage (whether a case is reviewed at all, which collapses from 100.0% to 65.6% under load, p < 0.001). Blogger's Review: This study suggests that using a multi-agent pipeline may not reduce bias, and audit capacity is crucial for detecting bias. Therefore, further research is needed to optimize the audit process to reduce bias.

Original Source: https://arxiv.org/abs/2608.06949

Next: None
[h] Back to Home