In large language model agents, substantial wall-clock time is often spent waiting for tool call results. Tool-call speculation can hide this latency by predicting and pre-executing the agent's next tool call, but existing speculators are typically separate draft models or cached traces that misalign with the deployed agent's behavior. We identify this speculator-agent gap and demonstrate that the target agent itself is a strong next-call speculator. This points to a simpler design: unifying the agent and speculator within the same model.
In this paper, we introduce the self-speculating agent, a single model that operates in agent mode to solve tasks while predicting its next tool call from partial trajectories in speculator mode, fully reusing prefix KV cache. To enable this dual-mode agent without degrading performance, we propose a joint agent-speculator reinforcement learning method that derives speculation targets from the agent's own rollouts and alternates agent and speculator updates.
Across agentic search QA and conversational tool-use agentic tasks, our method improves average next tool-call Hit@1 from 44.1 to 61.2 for Qwen3-4B and from 48.9 to 66.3 for Qwen3.5-4B, while preserving agent task success.
Blogger's Review: The proposed self-speculating agent effectively integrates speculation and agent functionalities via reinforcement learning, simplifying model design while significantly enhancing the accuracy of tool calls. This innovative approach holds great potential for practical applications and warrants further research and exploration.