Production multi‑agent systems constantly replace agents under the assumption that any agent capable of a role is interchangeable with any other. We empirically test this assumption. For each setting we independently form eight teams, all derived from the same base model and solving identical tasks. Each agent maintains a private notebook over ten formation episodes. Afterwards we swap role‑matched agents between teams and evaluate the impact on held‑out tasks. A placebo condition reproduces the disruption caused by a roster change without actually swapping occupants. Results show that swaps have little effect on task scores but increase communication per unit of progress by 16%–63%. In Hanabi, a swapped agent is more costly than a completely inexperienced one, indicating interference from conventions learned with its former partner. In Collab‑Overcooked, when the agenda‑setting agent is replaced, most of the extra communication originates from the agent that remains. Three ablations—varying base models, decoding temperature, and formation length—reveal that the swap penalty moves in tandem with how far independently formed teams diverge. Greedy decoding reduces both effects; doubling a team’s history raises both. In summary, agents are more fungible in task outcomes than in coordination efficiency, and longer formation histories amplify the negative impact of swaps.
Review