We observe that current research in explainable artificial intelligence (XAI) largely optimizes a set of proxy metrics, seeking higher scores on these imperfect measures. Because the metrics themselves are flawed, they fail to capture the true quality of methods, and the metric‑driven focus often sidelines the rationality and interpretability of the methods themselves. This motivates a new line of work—interpreting XAI methods, which we humorously call XXAI, and this paper is a contribution to that line.
The target of our study is Generic Attention‑model Explainability (GAE), a widely adopted XAI technique for Transformers. By dissecting its internal workflow we reveal that GAE essentially treats attention weights as direct explanations, ignoring the cumulative nature of attention updates and the nonlinear feature interactions inherent in Transformer layers. Consequently, GAE yields biased interpretations, especially for models that employ special tokens.
To address these issues we introduce Cumulative Asset Holdings (CAH), an interpretation method that merges process‑based and feature‑based perspectives under an economic zero‑sum game framework. Each attention update is modeled as an asset transfer, and the accumulated holdings quantify the contribution of inputs to outputs. CAH naturally handles special tokens such as CLS or SEP, overcoming the limitations of existing approaches.
The paper also demonstrates a model simplification strategy and an additive‑operation analysis for Transformers, offering a blueprint for uncovering the core logic of other XAI techniques.
Review