HarnessFlow is a framework that optimizes complete executable harnesses under a fixed model and task environment while operating with a limited candidate‑evaluation budget. It organizes all candidates and their evaluation histories into a refinement tree and employs a Monte Carlo Tree Search (MCTS)‑based policy to balance exploitation and exploration. Selected harnesses are handed to a coding agent, which refines them using archived code, execution feedback, and a library of reusable harness mechanisms.
Across six benchmarks covering agentic tasks, code generation, and web interaction, HarnessFlow achieves the highest mean test score with only 15 refinement attempts per benchmark. The optimized harnesses attain an average score of 69.48%, improving over the initial harnesses by 10.89 percentage points and over Meta‑Harness by 3.95 points, while also reducing average per‑task execution cost compared to hand‑engineered baselines.
These results indicate that structured harness search can boost performance and cut computational overhead simultaneously, offering a promising route toward automated design of efficient LLM agents.
Review: Overall, HarnessFlow demonstrates how coupling search algorithms with code generation can yield substantial performance gains under tight evaluation budgets.