Some defects in AI‑generated studies are observable directly from their artifacts, while others require knowledge of what was approved before execution. To address this, we introduce study contracts that bind declared experimental choices, run obligations, and claim scope to recorded execution evidence, and we separate contract‑relative verification from scientific truth. A diagnostic using eight self‑crafted clean/mutated pairs illustrates the information boundary. A deterministic checker applies a registered, fault‑specific rule to both approved and executed objects and successfully detects all eight registered mutations. Across eighteen recorded judge aliases, each receiving metadata‑filtered packages without pair context or the registry‑selected fault label, we evaluated 144 mutated cases: 104 received defect flags, 32 abstained, and eight terminated with errors, with no explicit clean decisions on mutated cases. Some packages retained approval and execution fields, including digests. The prompt instructed judges to abstain when evidence was insufficient. These results characterize a deliberately information‑asymmetric development setting, but they do not isolate the effect of authoritative information from differences in task specification or rule selection, nor do they constitute a comparative study of verifier quality or agent reward hacking. We identify the need for full‑information comparisons, legitimate‑adaptation controls, and closed‑loop agent evaluations to test whether contract checks improve useful, compliant completion under optimization.
Review