NeFut Logo NeFut
Admin Login

[CS.AI] Asclepius: An Adaptive Harness for Long-Horizon Clinical Agents

Published at: 2026-09-15 22:00 Last updated: 2026-09-16 00:22
#Machine Learning #LLM #Artificial Intelligence

LLM agents are usually evaluated on short, single‑task trajectories, yet real‑world deployments must run for hours under continuous time and resource pressure, exposing a different class of failures. We employ the Clinical Environment Simulator (CES) as a testbed, where an agent manages an entire emergency‑department shift under sustained contention. Long‑horizon execution failures become measurable in a single rollout through a structured, multi‑dimensional grading scheme. Current agents typically arrive at the correct diagnosis but fall short on delivering complete and timely critical actions, revealing an execution gap. We attribute this gap to three long‑horizon failure modes, each quantified by a per‑trace counter: instruction‑adherence drift, treatment incompleteness, and a severity‑equity timeliness gap. To address them, we introduce Asclepius, an adaptive harness that rewrites the operating manual between shifts based on trace‑level feedback, leverages an external clinical skills library for high‑stakes regimen knowledge, and employs three isolated sub‑agents that partition per‑turn decisions across the patient queue. On held‑out batches never seen during harness evolution, Asclepius improves critical‑action correctness by 22% (p = 0.024) while preserving diagnostic accuracy, with consistent gains across five LLM judges from three model families; on the full ten‑batch set, improvements reach 25% on critical actions and 13% on timeliness. The three failure modes form a coupled bottleneck, and decisive reductions appear only when all three components act together.

Review: Asclepius demonstrates that a self‑evolving manual, an externalized skill repository, and decision‑making sub‑agents can jointly close the long‑horizon execution gap, offering a practical route toward reliable clinical LLM deployment.

Original Source: https://arxiv.org/abs/2609.13543

[h] Back to Home