Language model agents operating over long interaction streams must adapt online to distribution shifts, which calls for continual learning. Existing agent harnesses either evolve their states solely via forward information flow or depend on extra replay or validation data for optimization. To tackle this, we introduce ReHarness—a recurrent harness framework for agentic continual learning that, under a strictly online streaming protocol, augments forward read/write with backward revision. The key idea is to treat the trajectory generated after each instance as feedback and use it to revise the reflections retrieved for that instance, thereby propagating feedback backward through the textual harness state.
Across three continual‑learning benchmarks and three language‑model backbones, ReHarness consistently outperforms strong raw‑context baselines, memory‑ and reflection‑based methods, and prompt‑optimization approaches. Further analysis shows that ReHarness produces source‑grounded, meaningful revisions, corrects over‑generalized or outdated experience under distribution shifts, and improves both stability and plasticity during continual learning.
Review