The paper examines the notion of power in AI safety. Optimal agents tend to acquire power across many objectives (instrumental convergence), which can gradually disempower humans in socio‑technical systems. A formal, quantitative measure of power is therefore required. Two measures in Markov games are considered: information‑theoretic empowerment, the channel capacity from an agent’s actions to future states; and capability‑based individual power, defined by the probabilities that a human would achieve each goal in a candidate set. We show that they intersect at goal revelation. Using Fano’s inequality, individual power yields a floor $B(s)$, a lower bound on the mutual information between the human’s latent goal $G$ and the realized trajectory $\xi$: $\mathbb{I}(G;\xi \mid s)$. Empowerment caps revelation from above, giving the chain
$$B(s) \le \mathbb{I}(G;\xi \mid s) \le \mathbb{I}(\mathbf{A}_h \to \xi \mid s) \le \mathfrak{E}_h(s)$$
Each quantity can be large while the one below it remains zero, exposing two failure modes. An assistant that maximises empowerment may prefer an interface that offers broad influence over goal‑irrelevant degrees of freedom rather than one that lets the human attain every possible goal. An assistant that maximises revelation cannot distinguish a functional interface from one rewired so that every goal leads to the wrong outcome, because relabelling goals leaves the mutual information unchanged while zeroing all attainment probabilities. Hence positive guarantees arise only from the attainment end of the chain. We propose a satisfaction floor, computable from two statistics of interaction logs, to monitor the power retained by users in a deployed system. Simulations in random environments show the three quantities are correlated but far from interchangeable.
Review: The work establishes a hierarchical power framework and highlights that empowerment or revelation alone are insufficient; concrete attainment guarantees are essential for preserving human control.