NeFut Logo NeFut
中 Admin Login

[CS.AI] Frustratingly Easy Agentic RAG via Portable Skills: A Case Study in Customer Support

Published at: 2026-10-15 22:00 Last updated: 2026-10-18 08:05
#AI #Machine Learning #LLM

Enterprise workflow automation usually requires retrieving a complete and accurate set of relevant information from large operational document corpora (SOPs, policy documents) for a given scenario or request. In customer support, human specialists must determine the correct sequence of resolution steps for each case, demanding precise grounding in policy documentation. Production systems often rely on complex document chunking, indexing, and retrieval pipelines, layered with learned re‑rankers, query rewriters, and custom agentic RAG infrastructure. We experimented with a simpler alternative: a portable skill consisting of a natural‑language instruction, lightweight tools (a retrieval API, format‑checking scripts), and a corpus enriched with an LLM‑generated navigation layer that acts as a structured “Wikipedia‑style” index over the underlying documents. Implementation required only 1‑2 engineer days, is portable across agentic harnesses (e.g., Claude Code), and performed strongly on the customer‑support workflow generation task, achieving 99.2% procedural equivalence and 83.3% evidence sufficiency. The only drawback is latency—about 90 seconds per case, roughly ten times slower than our production assistant (8‑9 seconds), so there is no latency advantage. Our claim is scoped to settings with a mature, well‑curated corpus, latency‑tolerant workflows, and human review of every case. In such environments the skill replaces a complex RAG stack, and maintenance shifts from model retraining to editing text and data.

Review

Original Source: https://arxiv.org/abs/2610.14428

[h] Back to Home