LLM‑based AI agents handle user requests by iteratively reasoning and invoking tools, typically calling remote LLM APIs while running tool containers locally. This execution model makes service optimization hard because latency, resource demand, and container bottlenecks intermix across requests. The current agent ecosystem largely ignores these resource dynamics, leading to substantial waste of valuable resources.
We study three representative tasks—retrieval‑augmented question answering, web search, and code generation—to examine how concurrent processing of multiple requests and tasks mixes resources, and we characterize latency as a function of those dynamics. Measurements reveal that the same tool can exhibit markedly different resource behavior across tasks, and concurrent requests expose task‑specific bottlenecks in CPU, disk I/O, and memory. Moreover, faster LLM responses or more CPU cores do not always speed up agents.
Guided by these observations, we propose two optimizations that exploit task‑aware resource dynamics: a CPU‑aware tool admission policy and a task‑aware CPU allocation scheme. Experiments show that latency for CPU‑sensitive tasks improves by roughly 5.4×, and the average latency across multiple tasks drops by about 32% compared to native agents.
Review