LLM‑driven multi‑agent systems (MAS) have shown strong abilities in tackling complex problems across diverse domains. Recently, dynamically orchestrating agents during execution has become a key research direction, yet existing approaches suffer from limited composition, misaligned dependencies, and rigid scaling, which hampers adaptability to evolving reasoning needs. To overcome these issues, we reformulate MAS design as a partially observable Markov decision process (POMDP), allowing both the composition and scale of the system to be determined dynamically by the decision process.
We introduce DHCG, a framework that coordinates three modules—Planner, Worker, and Generator—to progressively build a hierarchical collaboration graph from scratch based on the query and ongoing execution feedback. At each step, guided by the feedback, the Planner creates a set of distinct, complementary roles tailored to the current reasoning requirement and selectively routes relevant information to each role. The Planner can also terminate the graph early when the task is satisfied or expand it further when additional reasoning is needed. To make the Planner’s decisions more effective, we propose action‑aware preference optimization, which biases the Planner toward constructing more useful hierarchical graphs.
We conduct systematic evaluations on code generation, mathematical reasoning, and domain‑specific reasoning benchmarks. DHCG achieves the best average performance among all baselines, improving over the single‑agent baseline by 13.06 points and outperforming both static and dynamic MAS baselines by 2.77‑8.02 points. Additional experiments demonstrate strong generalization across different Planner backbones and unseen Worker models.
Review: By casting MAS as a POMDP and introducing a dynamically built hierarchical collaboration graph, DHCG enables flexible expansion and early convergence during reasoning, markedly boosting the adaptability and overall effectiveness of multi‑agent systems.