Evaluator‑guided reasoning faces a local control decision at each step: trust the current answer, fall back, or continue inference. We model this as a compute‑aware three‑action controller. Terminal actions are scored by predicted answer success, while expansion actions are scored using a rollout‑based continuation‑success surrogate, with compute cost explicitly priced.
A local action‑gap analysis characterizes when score errors can flip a decision, and an expected regret bound accounts for accumulated errors along the realized expansion path. Guided by this analysis, we learn and calibrate confidence estimates for each action and add local uncertainty penalties for ambiguous choices.
On held‑out width‑expansion and answer‑revision tasks, the controller improves observed utility over calibration‑selected fixed policies while reducing normalized online compute by about 30% and 16%, respectively. Across four Qwen3‑8B seeds, compute is lower than the fitted Q baseline, with utility differences small and mixed.
The results frame evaluator‑guided reasoning as resource allocation under uncertainty rather than merely a stronger reasoner.
Review