NeFut Logo NeFut
中 Admin Login

[CS.AI] AgentOrch: How Close Can Global Orchestration Come to Step-by-Step Interaction?

Published at: 2026-10-27 22:00 Last updated: 2026-10-29 00:58
#AI #Machine Learning #Artificial Intelligence

Enterprise workflow agents usually follow a step‑by‑step interaction (Step-by-Step) pattern: the agent performs an action, observes the outcome, and then calls a large language model (LLM) to decide the next step. Global orchestration (Global) instead generates a complete executable workflow in advance and runs it without further LLM calls, dramatically reducing inference overhead, yet it remains less accurate than the step‑by‑step approach.\ \ We introduce the AgentOrch benchmark, which directly compares Global orchestration with Step‑by‑Step interaction on 650 enterprise SOPs and 3,220 runtime variants under matched tools, environments, and evaluation protocols. The results show that Step‑by‑Step is still more accurate for every evaluated model, but the accuracy gap has shrunk by about 79% from early models to the strongest recent model.\ \ We identify three avenues for further narrowing the gap:\

  1. Incremental workflow construction – building the global workflow incrementally without runtime feedback improves Global accuracy by an average of 9.98 percentage points (pp).\
  2. Workflow revision with execution feedback – using actual execution results to revise the complete workflow benefits all models, including a 6.13 pp gain on held‑out runtime variants.\
  3. Targeted training on complete workflows – fine‑tuning five models on full workflows yields an average accuracy increase of 12.18 pp.\ \ Although a gap remains, Global orchestration offers a promising path toward enterprise agents that achieve higher accuracy with fewer LLM calls.\ \ Review
Original Source: https://arxiv.org/abs/2610.33375

[h] Back to Home