Inference‑time harnesses—external procedures that call a model repeatedly while maintaining state—can substantially boost model capabilities.
We formalize a harness as an arbitrary wrapper $\mathcal{A}$ around a base model $M$ and attempt to distill a student model from the harness's execution traces.
Empirically, even with very simple scaffolds and small models, the inference‑time harness markedly outperforms the distilled student.
We then prove that this gap stems from a fundamental representational limitation: when the per‑call primitive is a trivial $\AC^0$ lookup, the harness can compute the closure of that primitive, which is already $\NL$‑complete. Under the widely believed conjecture $\NL \not\subseteq \TC^0$, a fixed‑depth forward pass cannot represent this closure.
The theory suggests that closing the gap requires expanding the student’s representational capacity relative to the task—e.g., enlarging the model, simplifying the task, or lengthening the chain‑of‑thought—and our experiments confirm these remedies.
Finally, we scale to an agentic harness on competition‑level mathematics, where the same gap reappears. Overall, a harness confers a profound increase in representational power to the base model, a gain that training a same‑class student on its traces cannot transfer back into the weights.
Review