NeFut Logo NeFut
中 Admin Login

[CS.AI] Persistent Retrieval for Conversational Agents: An Efficiency Study

Published at: 2026-11-06 22:00 Last updated: 2026-11-07 12:32
#algorithm #AI #Machine Learning

Conversational agents may fetch information themselves or delegate the task to a sub‑agent. This choice influences how retrieval history is stored, what context each model call receives, and what must be re‑retrieved for follow‑up questions. We compare three architectures: skill‑based retrieval inside the main agent, a non‑persistent sub‑agent that returns a synthesized finding, and our persistent sub‑agent that keeps retrieval history across turns and returns raw evidence to the main agent. The persistent sub‑agent also allows the main agent to ask for corrections from the same sub‑agent. Experiments evaluate the designs on enterprise questions asked independently, within shared conversations, and as dependent follow‑ups across five multi‑turn settings. Compared with skill‑based retrieval, the persistent approach reduces recorded model tokens by 43.3%, lowers estimated API cost by 45.0%, and cuts average response time by 9.1%. Compared with non‑persistent retrieval, it shortens response time by 15.6 seconds per question while using 65.1% more tokens and incurring 29.1% higher estimated API cost. In summary, the persistent sub‑agent shifts from generating synthesized answers to maintaining retrieval state and supplying evidence that the main agent can directly evaluate, improving both efficiency and cost.\n\nReview: Persistent retrieval demonstrates a clear advantage in reducing token usage and latency, offering a practical optimization for real‑world conversational systems.

Original Source: https://arxiv.org/abs/2611.07899

[h] Back to Home