Abstract
Objective: Concept bottleneck models route prediction through interpretable intermediate variables, and their validity is normally judged by how accurately those variables are predicted. We question whether that judgment is sufficient, using left ventricular volumes as the concepts underlying ejection fraction estimation from echocardiographic video.
Methods: A video transformer encoder was trained on a publicly available echocardiography dataset. End-systolic and end-diastolic volumes formed a concept layer from which ejection fraction was computed analytically, with no residual path to the output. We compared training under an ejection fraction objective alone against training with additional supervision of the volumes in millilitres, and evaluated both on 1276 held-out studies.
Results: The concept bottleneck did not increase ejection fraction error relative to direct regression, at 6.89 against 7.13 mean absolute error. Without volume supervision, however, the spread of predicted volumes collapsed to 0.1 millilitres against reference spreads of 35.7 and 45.7 millilitres, while correlation was partly preserved. We show that this follows from an invariance property of the objective: ejection fraction is a ratio and is unchanged when both volumes are rescaled, so the loss determines the concept layer only up to scale. Supervision in absolute units reduced volume error from 89.8 to 25.8 millilitres at a cost of 0.4 in ejection fraction error.
Conclusion: Concept accuracy alone can conceal a concept layer that carries no physical scale.
Significance: Interpretable intermediate variables in clinical models should be validated against the invariance structure of the training objective, not only against prediction accuracy.
Blogger's Review: This paper explores the effectiveness of concept bottleneck models in echocardiography applications, showing that incorporating volume supervision significantly enhances model accuracy. It emphasizes the impact of loss function invariance on model outputs, providing a fresh perspective on assessing the interpretability and reliability of clinical models.