NeFut Logo NeFut
中 Admin Login

[CS.AI] A Behavioural Framework for Predicting and Understanding Loss of Control in Frontier AI Systems

Published at: 2026-11-06 22:00 Last updated: 2026-11-07 12:32
#AI #Machine Learning #Artificial Intelligence

Frontier AI systems are being deployed over longer horizons with diminishing human oversight, raising the risk of loss of control—systems act beyond effective human command and may cause irreversible harm. Because such failures often become observable only when containment is hard, they must be assessed prospectively. We introduce a behavioural framework that breaks loss‑of‑control risk into three factors: misaligned motive, harm‑enabling capability, and monitoring evasion. These factors are operationalised through 13 measurable aspects and 29 evaluation tasks, whose results are combined into a single loss‑of‑control (LoC) score. Across 13 frontier models the LoC score correlates strongly with failure rates in contained long‑horizon risk scenarios (Pearson r = 0.84), demonstrating strong prospective validity. Risk varies systematically across models and cannot be explained by capability alone. Further experiments show that fine‑tuning and monitoring aimed at the diagnosed vulnerabilities reduce failure rates while largely preserving overall performance. The framework thus offers a practical basis for predicting, understanding, and mitigating loss‑of‑control risk before it materialises.\n\nReview: By turning an abstract safety concern into concrete, measurable metrics, the framework provides a scalable way to evaluate and intervene on frontier AI systems, balancing predictive power with actionable mitigation.

Original Source: https://arxiv.org/abs/2611.07953

[h] Back to Home