Achieving early and timely diagnosis and treatment for diseases is a major challenge. Recent applications of machine learning (ML) algorithms trained on patient data have shown promise in predicting patient health states. However, a frequent challenge faced when applying these ML algorithms is that not all clinical variables (features) needed for prediction tasks are available at any given time.
We define the concept of Full-Feature-Capacity (FFC) to refer to the prediction performance when algorithms utilize all features on which they were trained. We then introduce Feature Sufficiency Analysis (FSA), an analysis for determining whether a subset of clinical features needed by an AI model is sufficient to achieve FFC. FSA estimates the underlying distributions of missing variables conditioned on available features. It provides a patient-specific assessment of whether the existing set of measured features achieves FFC. If so, there’s no need for further input acquisition, allowing for ML-based predictions.
We provide two case studies: predicting the need for postoperative prolonged ventilation in patients recovering from heart surgery and predicting 10-year mortality in an outpatient cohort. FSA also offers a clinically interpretable feature-ranking methodology based on prediction sufficiency, identifies intrinsically hard-to-predict patient populations, and has the potential for cost-aware optimization of clinical data acquisition. FSA provides a generic computational approach to determine whether incomplete clinical information is sufficient to support trustworthy AI-assisted clinical decision-making, thereby facilitating the prospective deployment of healthcare AI systems across diverse clinical settings.
Blogger's Review: The Feature Sufficiency Analysis introduced in this paper provides a crucial theoretical foundation for the application of healthcare AI, particularly in scenarios with incomplete data. By incorporating clinical interpretability, FSA not only enhances the reliability of models but also offers greater support for real-world medical decisions. Its potential for cost optimization also presents a new approach to the rational allocation of medical resources.