Tool‑using agents receive new interaction data after each policy update, which can render previously estimated action credits stale. Recomputing them from scratch requires many additional tool calls and environment interactions, making repeated updates costly.
The key observation is that a change in action value does not necessarily alter the decision. As long as the policy‑induced drift is too small to overturn the existing action ranking, historical credit remains useful.
To quantify this effect, we introduce pairwise branch sensitivity, which measures how strongly a policy update affects the downstream branches that distinguish two candidate actions.
Building on this, we derive a first‑order anchored credit‑transport estimator that updates historical credit using old interventional trajectories, avoiding the need for fresh sampling.
We then propose the Decision‑Sufficient Credit Gate (DSC‑Gate), which decides whether to reuse, transport, or resample credit based on branch sensitivity.
Experiments show that branch sensitivity explains credit drift far better than global policy distance. With sufficient historical data, credit transport reduces estimation error, and its benefit to decision making is concentrated on updates that affect action‑distinguishing branches. On an independent test set, DSC‑Gate changes mean regret by only +0.00004 relative to a gap‑based gate while cutting mean new tool steps from 472 to 286, a 39.4% reduction. The same pattern appears after a real tool‑agent parameter update.
Conclusion: Agents do not need to recompute action credit after every policy update; much of the historical evidence can be reused or cheaply corrected, substantially lowering the extra interaction required to keep decisions up‑to‑date.
Review