Although large language models (LLMs) achieve impressive capabilities, their reliance on cloud‑scale infrastructure brings latency, privacy, connectivity, and high computational cost issues.
Many agentic workloads contain repetitive, narrowly scoped subtasks that can be handled more efficiently by specialized small language models (SLMs). Yet the limited capacity and context windows of SLMs hinder long‑horizon reasoning and interaction‑heavy orchestration strategies such as iterative verification or debate.
This motivates a non‑interactive paradigm: heterogeneous SLMs generate candidate solutions independently, and a router selects among the cached samples without further model calls.
To study this systematically, we introduce OrchSLM, a routing framework that unifies existing non‑interactive orchestration methods and exposes their core design choices as controllable knobs. Systematic probing shows that orchestration behavior is shaped by task structure, model‑pool composition, and multi‑agent consensus.
Review