NeFut Logo NeFut
中 Admin Login

[CS.AI] Conformance as a Distribution: Delegation Contracts, Error Budgets, and the Allocation of Evaluation Authority in Use-Based AI Regulation

Published at: 2026-10-27 22:00 Last updated: 2026-10-29 00:58
#algorithm #AI #Machine Learning

Use‑based AI regulation assigns obligations according to deployment risk but lacks a concrete evidentiary object. This work treats a benchmark score as an intrinsic model property and defines deployment conformance as the failure rate under a declared distribution of contexts, bounded by scope and expiry. We adopt the earlier delegation‑contract framework (frozen configuration, scoped claim, executable revision rule) and analyze it as a strategic game among a deployer, an auditor, and an enforcement point.\ \ First, we ask whether a declared context mixture can be made truthful. By imposing a proper‑scoring penalty on the declared mixture, the excess failure of a best‑responding deployer is bounded by $O(\lambda^{-1/2})$ in the penalty weight.\ \ Next, we address audit bias. A commit‑reveal audit drawn from live traffic is unbiased by construction and grades the evaluator using the same sample.\ \ For error budgets we propose two mechanisms: a halving error budget drives certification coverage down to $(1-p)^n$, while an $\alpha$‑investing scheme sustains coverage under a marginal false‑certification guarantee.\ \ Regarding subgroup guarantees, we show that audit size scales as $1/m$ with subgroup mass, enabling aggregate audits to support per‑subgroup assurances. Finally, we examine an incident metric that counts only system failures and ignores fallback costs; this metric always prefers withdrawal at any fallback cost, even when total loss ranks arrangements oppositely.\ \ Five preregistered synthetic studies validate each claim; no language model or production system is evaluated. Enforcement points sit at distribution, capability‑access, monetization and risk‑transfer choke points, expiry of the conformance record constitutes a standard of care, and conformance‑as‑a‑rate contrasts with the EU AI Act’s static conformity assessment.\ \ Review

Original Source: https://arxiv.org/abs/2610.33112

[h] Back to Home