NeFut Logo NeFut
Admin Login

[CS.AI] When Can a Machine Trust a Statute? A Survival Certificate for Machine-Extracted Legal Logic

Published at: 2026-09-03 22:00 Last updated: 2026-09-04 02:14
#AI #Machine Learning #Legal

This paper investigates how statutory noise introduced by machine parsers affects formal logical inference. Using Missouri statutes as a testbed, two independently written extractors were compared on the detection of numeric thresholds, revealing a false‑negative rate of 0.43. To determine which implications survive such noise, we construct a passive survival certificate for the Duquenne‑Guigues implication basis. The procedure consists of:

  1. Measuring per‑attribute disagreement between the two extractors;
  2. Re‑sampling the disagreement rates in 1,000 Monte Carlo trials and rebuilding the implication basis each time;
  3. Computing a one‑sided Wilson 95% lower bound for each implication and certifying it when the bound reaches at least 0.95. Certified implications include premise spans and a minimal counterexample.

Experiments cover 29,365 Missouri sections and 502 Indian central‑Act sections. All preregistered held‑out gates (10 statute families across 7 titles exact; 16 across 11 titles with 5% tolerance) passed. Under a globally deployed error model, however, 93.2% of held‑out chapters fell below the informativeness floor. A 2×2 factorial analysis attributes this drop to calibration‑rate transfer rather than selection bias.

The certificate is usable but fragile: it must be deployed per‑chapter with calibration or error tolerance. All code, data products, and the audit trail—including a retracted claim—are released.

Review: The study offers a quantifiable framework for assessing the reliability of machine‑extracted statutory logic. Although highly sensitive to the error model, it marks a significant step toward verifiable legal AI.

Original Source: https://arxiv.org/abs/2609.01741

[h] Back to Home