NeFut Logo NeFut
Admin Login

[CS.AI] MetaRAG: Belief-Action Aligned Policy Optimization for Agentic Retrieval-Augmented Generation

Published at: 2026-08-26 22:00 Last updated: 2026-08-29 12:04
#AI #Machine Learning #LLM

Agentic retrieval‑augmented generation (RAG) requires language models to decide whether to keep searching or to answer directly. Existing reinforcement‑learning methods rely on external supervision and ignore the model’s internal belief about the sufficiency of the current evidence. To tackle this, MetaRAG reformulates the quality of the search decision as belief‑action alignment and introduces a corresponding policy‑optimization framework.\ \ MetaRAG employs Verify‑first Action Generation, which triggers an explicit verification step before each actual action, and Internal Belief Probing, which estimates the policy model’s own answerability belief from the same question‑history context. Using these, a consistency reward is computed and further gated by answer correctness, preventing reinforcement of trajectories that are internally consistent yet wrong. The belief probe is used only during training, incurring no overhead at inference time.\ \ Experiments on seven public QA benchmarks demonstrate that MetaRAG consistently improves the accuracy‑efficiency trade‑off over strong RL‑based agentic RAG baselines. The gains transfer to deep‑research settings, various optimizers, and multiple model backbones.\ \ Blogger's Review: By explicitly modeling the model’s self‑belief, MetaRAG cleverly couples internal consistency with external correctness, offering a promising direction for making RAG systems more efficient and reliable.

Original Source: https://arxiv.org/abs/2608.24214

[h] Back to Home