Deploying large language models in production often requires constraining outputs to structured formats such as JSON or XML. Prior work labeled the resulting drop in accuracy as a “structure tax”. We revisit this claim by evaluating a wide range of models, datasets, and schemas, measuring task accuracy, confidence calibration, and hidden‑state geometry.
Our experiments show that the tax is not caused by structure itself but by schema design. A reasoning‑first field ordering (reasoning first, answer later) matches or even exceeds free‑form accuracy, whereas an answer‑first ordering leads to steep accuracy drops, especially in smaller models. Sensitivity to format scales inversely with a task’s inherent structural constraints, and schemas that preserve the reasoning order also improve calibration; CKA analysis reveals greater separability between correct and incorrect representations in middle transformer layers.
Thus, properly designed structured formats can match or surpass free‑form performance. The critical question shifts from “whether to structure” to “how to structure” for optimal reasoning preservation.
Review: The paper provides solid empirical evidence that schema layout, not mere structuring, determines performance loss, offering clear guidance for real‑world LLM deployment.