Auditing the outcomes of large language model (LLM) training is difficult because model weights and training data are usually private. Providing a cryptographic proof of the full training process at Transformer scale is practically infeasible. To address this, we introduce zkLLMPoT, a zero‑knowledge framework that certifies auditor‑defined properties of a trained checkpoint via forward evaluation rather than by verifying the optimization trajectory.
zkLLMPoT operates in two phases. In the first phase, the trainer fixes the architecture and commits the model weights; the auditor then selects challenge sequences, preventing the trainer from altering the checkpoint in response to the audit data. In the second phase, the trainer proves the objective value achieved by the committed model on those sequences. This formulation makes certification cost independent of the number of training iterations, while keeping weights hidden and avoiding any access to private training data.
From a technical standpoint, we build on sumcheck and lookup arguments to certify the core Transformer operators (e.g., attention, feed‑forward layers) and support next‑token loss as well as task‑specific audit objectives. Benchmarks across four model families show operator‑level proving times of 41‑59 seconds for 1.1‑1.5 B‑parameter models and 131 seconds for a 13 B‑parameter model; verification stays below half a second for a sequence length of 512.
Review: By shifting the proof target from the entire training process to forward inference, zkLLMPoT dramatically reduces the cost of zero‑knowledge auditing, offering a practical way to verify training quality while preserving model privacy.