NeFut Logo NeFut
中 Admin Login

[CS.AI] Towards Portable Specification of the Agent Harness

Published at: 2026-11-03 22:00 Last updated: 2026-11-05 00:45
#AI #Machine Learning #LLM

Large language models operate solely through the surrounding infrastructure, referred to as the harness. The harness determines what a model can achieve and how it may fail, yet no standard definition exists. To bridge this gap, we decompose the harness into eight functional components and propose a portable specification framework that records which model runs in which harness. Such a record is essential because different harness configurations can cause several‑fold variations in reported scores for the same model, and a large portion of an agent's attack surface resides in its harness. Existing model cards and agent cards provide release and operational details but do not capture the sequence of effective configurations within a single run. Harnesses may change across releases, across runs, and even within a run when agents modify their own tools and settings. Enacted AI laws impose documentation duties on covered systems, but they do not prescribe the unified harness record proposed here. Consequently, the harness must be explicitly described and versioned as part of the system.

Review: A clear harness specification improves reproducibility, mitigates evaluation bias caused by environmental differences, and offers a traceable technical basis for compliance audits.

Original Source: https://arxiv.org/abs/2610.40917

[h] Back to Home