AI control is a family of techniques aimed at preventing AIs with malicious goals from subverting their operator's intent. Traditional AI control often studies a single agent in one trajectory, but real deployments involve many agents operating over shared infrastructure, with severe risks such as model-weight exfiltration and training-run poisoning likely requiring multiple agents acting in concert.
We initiate the empirical study of multi-agent AI control, formalizing distributed attacks where several agents jointly pursue a malicious goal.
We developed FakeLab: a synthetic AI-lab codebase comprising 9 services, 86 benign tasks, and 4 attack objectives. We evaluate single agent monitoring against distributed attacks, varying the number of agents, their coordination, model capabilities, and precise monitoring configurations.
Our central finding is the fragmentation effect: as more agents coordinate to attack, per-agent monitoring becomes less likely to catch any of the attackers. Secondly, this is not driven by the ratio of benign to malicious code; we argue it is likely dependent on model capability.
Third, an explicit planner amplifies the fragmentation effect, raising the rate of attack completion by up to sevenfold and mildly sharpening executors' poor monitor-theory-of-mind. Finally, a weak 'trusted' monitor misses most attacks, whereas a stronger one reduces undetected success by more than an order of magnitude but remains imperfect.
We are happy to share FakeLab, our control setting, with safety researchers upon request.