The rapid adoption of LLMs in writing tasks has spurred the creation of tools that detect LLM‑generated text, yet most of these tools only measure the surface wording and ignore whether the underlying ideas originate from the model or the user prompt. This paper introduces a framework that quantifies the additional value a human contributes beyond what a language model could easily produce. The approach requires no training data or labels and never scores the document’s surface text, thereby avoiding stylistic confounders. The core idea is to extract the document’s content at progressively finer granularities, ask an LLM to reconstruct the document from each partial representation, and compare those reconstructions with ones generated from the task description alone. We call this framework Value Over Language Model (VOLM), which measures contribution relative to a replacement‑level document that an LLM could generate from the task description.
We instantiate VOLM for three domains—news articles, ICLR peer reviews, and argumentative essays—and evaluate its performance. Results show that VOLM separates human‑authored texts from matched LLM‑generated texts produced from generic task descriptions, while remaining largely invariant to content‑preserving transformations such as LLM‑based reconstruction and round‑trip translation. Further analysis reveals that more constrained content extractors reduce the residual differences between LLM‑generated and humanized text, highlighting the importance of disentangling informational content from stylistic variation.
We hope these findings encourage specialized implementations of the framework and broader research on assessing human contributions in LLM‑assisted writing.
Review: VOLM offers a label‑free, information‑centric evaluation method that lays groundwork for measuring genuine human value in human‑AI collaborative writing.