Reinforcement Learning with Verifiable Rewards (RLVR) offers reliable supervision for software engineering (SWE) agents via executable tests, yet binary rewards cannot differentiate trajectories that vary widely in debugging quality. This work investigates whether the current policy can evaluate its own trajectories and use those judgments to improve task performance without a separately trained reward model. To this end we introduce Behavioral Self-Rewarding (BSR), a reward‑refinement method that:
- Reuses multiple trajectories from the same task that share the same verifier outcome, comparing observable debugging behavior;
- Aggregates pairwise preferences into task‑local scores;
- Maps the scores to bounded auxiliary rewards, preserving the pass/fail ordering defined by the verifier while adding reward contrast.
BSR thus extracts additional supervision from already‑collected trajectories without requiring new environment interactions. Experiments on SWE‑bench Verified and SWE‑bench‑Live using Qwen3‑14B and Qwen3‑32B configurations show consistent improvements across all evaluated metrics, confirming that a policy’s own comparative judgments can serve as a complementary learning signal for training SWE agents.
Review: BSR leverages internal comparisons instead of an external reward model, achieving efficient reward refinement. It offers a promising avenue for enhancing the debugging capabilities of SWE agents and can be readily integrated into existing RLVR pipelines.