Reinforcement Learning with Verifiable Rewards (RLVR) relies on scalar outcome rewards obtained after interaction to guide learning. For group‑relative objectives, this signal disappears when all rollouts receive the same reward, even though their trajectories contain useful information about task requirements and failure modes.
To address this gap, we introduce prospective learning, which uses post‑hoc experience to supervise predictions made before interaction. We instantiate it as Self‑Retrospection Distillation (SRD). Intuitively, a completed trajectory reveals knowledge that would have been valuable beforehand and pitfalls to avoid; SRD distills this privileged hindsight into trajectory‑blind foresight predictions of the same policy. The foresight serves only as a training target and is not required at inference time.
Across ten tool‑integrated reasoning and long‑horizon tasks, SRD complements RLVR and self‑distillation baselines, delivering up to $24.2$ pp improvement in success rate. The benefit is most pronounced when reward contrast is scarce: even when $37\%\sim98\%$ of rollout groups have uniform rewards across model scales, SRD still extracts learning signals from sampled trajectories. In the 2B setting, where $98\%$ of groups are all‑failure, RLVR alone achieves $0.0\%$ success, whereas adding SRD raises success to $60.6\%$ under the same rollout budget.
These results suggest that post‑hoc agent experience is useful not only for evaluating or refining behavior but also for shaping predictive representations before any interaction occurs.
Review