Abstract
Reinforcement learning enables Agentic RAG systems to learn multi-turn search from verifiable outcome rewards, but all-zero rollout groups provide no comparative signal and may hide useful search behavior. We present EviBack, an evidence-constrained Teacher backoff that supplies auxiliary supervision to such groups while preserving verifiable Actor rewards. It separates evidence assessment from answer refinement, preventing reference answers from overriding evidence-insufficiency judgments.
EviBack implements a fully automated, end-to-end GPT-5.5-assisted APE pipeline, starting from a manually authored single-prompt dual-task Teacher, automatically partitioning and labeling rollout data, and performing ablation, task decomposition, evaluation, and selection to produce a gated two-stage Teacher. Compared with the manual design, the resulting Teacher improves downstream F1 and valid-answer rate while reducing search, duplicate queries, and forced termination.
Across seven open-domain QA benchmarks and three Qwen3 scales, EviBack improves F1 over Search-R1 and raises both single- and multi-hop macro F1. We guarantee that the code will be made publicly available at a later stage.
Blogger's Review: EviBack combines evidence constraints with teacher backoff to significantly enhance the effectiveness of multi-turn search, showcasing the potential of reinforcement learning in complex search tasks. This method not only optimizes results but also effectively reduces the number of redundant queries, indicating broad application prospects.