The paper observes that most AI risk work concentrates on capability risk – the danger that systems become overly powerful, autonomous, or misaligned with human values. In contrast, the organizational dimension receives far less attention, especially whether the institutions building these systems are prone to drift toward failure. The authors argue that such a drift is real. Regardless of how capable future AI systems become, the organizations that develop them face structural dynamics similar to those that preceded past major technological catastrophes. By examining the Challenger shuttle, the Three Mile Island accident, and the Boeing 737 MAX crashes, the study extracts common structural mechanisms that led to each disaster and maps them onto contemporary AI development. The findings suggest that existing safety infrastructure may offer less protection than assumed, as organizations can formally comply with safety procedures yet still produce catastrophic outcomes. The pre‑disaster phase of AI development is still underway, and the paper aims to make these dynamics legible while they can still be interrupted.
Review