Agentic LLM workflows consist of model turns interleaved with tool interactions, so their end‑to‑end completion time depends on both inference speed and the timing of turn releases. Most runtimes release a turn as soon as it becomes ready; under contention this eager policy can accumulate released but unfinished work, and once submitted the turns can no longer be reordered by the workflow‑level policy, increasing tail latency. We introduce a tail‑risk‑aware turn release scheduling method that jointly selects the next ready turn to release and determines how much released but unfinished work to retain. The approach optimizes a mean‑Conditional Value‑at‑Risk (mean‑CVaR) objective to capture the evolving tail risk of unfinished workflows, uses online estimates of turn workload to prioritize ready turns, and adapts the released‑work budget based on observed queue pressure. Evaluation uses real agent execution traces from software‑engineering tasks across multiple LLMs and varying workflow arrival rates. The method matches eager release under light load and substantially reduces the P95 of workflow flow time under contention, achieving up to a $3.50\times$ speedup.
Review