In the design of intelligent agents, the agent harness serves as a crucial control layer that transforms a base Large Language Model (LLM) into an executable agent, managing context, tools, orchestration, memory, decoding, and output handling. While harness design significantly affects agent behavior, most automatic improvement methods focus on narrower artifacts such as prompts, pipelines, or workflows, and deployed agents typically reuse a single global harness for all cases.
To address this limitation, we introduce MemoHarness, an adaptive harness optimization framework that learns from its own executions. MemoHarness decomposes the harness into six editable control dimensions and stores per-case diagnoses and distilled global patterns in a dual-layer experience bank. It adapts the learned harness to each test case using retrieved experience without requiring test-time labels, feedback, or additional search.
In our evaluation across shell-agent, code generation, and analytical reasoning benchmarks, MemoHarness outperforms the fixed harnesses we compared against and shows selective transfer to unseen suites and base models. Its additional context can also remain cost-competitive when much of the retrieved experience is cacheable. These results provide evidence that execution experience is a practical substrate for building agent harnesses that are more adaptive than a single static configuration, while broader claims about statistical robustness and component attribution are left for future work.
Blogger's Review: The introduction of MemoHarness presents a novel approach to adaptive optimization of intelligent agents, achieving significant performance improvements through experiential learning. Its dual-layer experience bank design also opens new possibilities for personalized applications of agents, warranting further investigation.