This work investigates whether off‑the‑shelf small language models (SLMs) can reliably handle microtasks that support a frontier large language model (LLM) planner. The microtasks studied are auto‑approving shell commands, writing to memory, selecting tools, and ranking past dialogue turns. We construct a benchmark of four fixed‑prompt tasks, each equipped with a threshold $\tau$ anchored to a cheap non‑LLM baseline and a confidence‑interval (CI)‑aware eligibility rule: a configuration passes only if its confidence upper bound exceeds $\tau$.
Sweeping Qwen3 models of 0.6B, 1.7B, 4B, and 8B under their best settings (FP16, greedy decoding, one frozen prompt, no tuning) yields 16 configurations (4 tasks × 4 models) and none meet the eligibility criterion. A log‑probability decision‑threshold diagnostic (for tasks T1, T3, T4) together with a context‑length/cascade probe (for T2) separates the failures into capability deficits and failures that could be remedied by adjusting the decoding threshold, defining four distinct regimes.
Quantizing the models to 4‑bit (RTN, GPTQ, AWQ) harms performance in a size‑dependent manner but does not produce any eligible configuration. The same gap appears on Llama‑3.x (all 12 configurations ineligible) and remains robust to variations in the anchor $\tau$ and to neutral prompt paraphrases—0 out of 112 eligible across the original and three paraphrased prompts per cell.
The practical implication is to place SLMs behind a baseline that already satisfies the CI‑backed threshold and to invoke the SLM only when the baseline fails. For instance, a 4B re‑ranker applied to a BM25 shortlist improves BM25 by $+0.047$ (confidence interval $[0.020, 0.073]$) yet does not itself achieve eligibility certification.
Review