AI‑enabled systems must stay reliable when their operating conditions change—such as input perturbations, distribution shifts, or alterations in external dependencies. This makes robustness auditing essential for judging whether the evidence supporting a robustness claim is sufficient. Designing a robustness‑auditing framework is difficult because the required capabilities of such a framework have never been consolidated.
To address this gap, we introduce meta‑requirements: higher‑order specifications that describe what a class of auditing frameworks should support. In design‑science research, meta‑requirements are typically derived from literature and domain experts. We performed a systematic literature review, filtering 2,890 records down to 74 relevant papers and extracting 43 meta‑requirements across 16 categories. These were then evaluated by 128 practitioners, yielding 5,504 ratings and 586 substantive comments. Overall, 81.20% of the ratings indicated agreement or strong agreement, and 41 of the literature‑derived meta‑requirements were classified as strongly supported.
Practitioner feedback refined the set and led to the addition of a requirement concerning external components and supply‑chain dependencies. The final collection comprises 44 meta‑requirements (43 quantitatively validated) spanning the same 16 categories. This validated set can guide organizations in implementing AI robustness auditing frameworks, assessing existing frameworks for completeness, and iteratively improving them.
Review