Scientific progress often relies on collaboration, yet most existing autonomous systems capture little of this. We investigate the impact of allowing multiple agents to communicate at test time via a shared directory, focusing on the ARC-AGI-3 benchmark that demands novel problem‑solving. Experiments show that a team of $k$ communicating agents (team@$k$) achieves a success rate comparable to $4k$ independent agents, and the advantage grows as $k$ increases, indicating a compound scaling effect. Crucially, some tasks that no single agent can solve become reliably solvable by a cooperating team.
In the polyomino packing task, communicating teams outperform the best@$k$ baseline and set a new top score. For MNIST classifier compression, a team produced a 1,957‑byte model that reaches 99.4% test accuracy, smaller than the best known human solution and better than any single‑agent result.
These gains are not unconditional. When compute is limited or clear progress metrics are absent, independent agents may outperform communication. Only under sufficient compute and explicit feedback does multi‑agent communication consistently yield stronger outcomes.
Review: Real‑time test‑time communication unlocks substantial collaborative gains at scale, especially for complex problems requiring breakthroughs, but its effectiveness depends on resource availability and feedback clarity.