Large language models excel on standard medical QA but degrade on harder reasoning tasks, and traditional multi‑agent debate often underperforms strong single‑model prompting. Existing systems select experts and retrieve knowledge solely from the question, ignoring gaps that emerge during reasoning. We introduce challenge‑aware adaptive recruitment: an external CHALLENGER monitors the evolving reasoning trace, pinpoints factual or expertise gaps, and triggers targeted knowledge retrieval or mid‑discussion specialist recruitment. Experiments on the MEDAGENTSBENCH hard subset (862 questions across nine biomedical datasets) show that upfront retrieval offers little benefit, while in‑between retrieval markedly improves performance; on a multi‑specialist backbone it boosts accuracy on 8 of 9 datasets and adds 6 percentage points over Challenger‑Only. Ablations reveal that both retrieval timing and gap‑specific query signals contribute, and a live web‑search study confirms the timing advantage persists across retrieval backends. For specialist recruitment, self‑triggered adaptation is unreliable, whereas Challenger‑triggered recruitment raises GPT‑4.1‑mini MedQA accuracy from 46 % to 54 %, with positive gains on all tested generalization sets. Naïvely combining retrieval and specialist recruitment degrades performance, highlighting the need to coordinate adaptive intervention channels. These findings demonstrate that when and why external resources are recruited is a pivotal design choice in multi‑agent medical reasoning.
Review