In long‑horizon games, a failed trajectory only signals that an agent needs intervention, but it does not tell how the external execution harness should change. Conventional approaches supply a fixed set of recovery rules without diagnostic evidence. To address this, we introduce KPHarness, a framework that adapts the external harness of a frozen vision‑language‑action model via knowledge‑probing co‑evolution.
Core mechanism:
- Recovery knowledge offers context‑specific intervention hypotheses that steer active probing.
- Probing outcomes verify when these hypotheses apply and provide evidence to refine both the recovery knowledge and subsequent probe selection.
- This reciprocal process enables the harness to improve not only which interventions to apply but also how to investigate failures more effectively.
KPHarness separates fast policy execution from a slow research loop that translates accumulated knowledge and probing evidence into bounded harness revisions. Each revision is evaluated against the incumbent under matched sandbox conditions, and is accepted only if the paired evaluation shows improvement.
Validated recoveries are stored in a knowledge bank for cross‑task reuse, while immediate interaction feedback and delayed episode‑success credit update action and probing policies during adaptation.
Experiments: On Minecraft tasks covering mining, combat, and crafting, KPHarness outperforms bare policy execution, a manually specified fixed harness, and a knowledge‑only evolution baseline with non‑adaptive probing, achieving higher task coverage. Evaluation on unseen tasks further demonstrates that even with the evolved harness, knowledge bank, and probing policies fixed, task completion rates improve.
Review: KPHarness integrates diagnostic evidence into harness evolution, achieving finer‑grained intervention and probing coordination, and offers an interpretable, reusable solution for automated adaptation in long‑horizon gaming scenarios.