In this paper, we address the problem of multimodal federated learning with missing modalities. Existing methods typically rely on an additional public dataset or perform naive feature synthesis based solely on the available modalities, which has its limitations. To address these shortcomings, we propose ProMoE-FL, a Prototype-conditioned Mixture-of-Experts framework for robust missing-modality feature synthesis in multimodal federated learning.
ProMoE-FL builds a global client-aware prototype bank that captures clinically meaningful modality priors across institutions. Our Mixture of Experts is conditioned on these prototypes and modality indices to enable direction-aware expert routing for dynamically synthesizing missing features. We conduct extensive quantitative and qualitative evaluations on four public chest X-ray datasets (MIMIC-CXR, NIH Open-I, PadChest, and CheXpert) and demonstrate that ProMoE-FL consistently outperforms state-of-the-art methods in both homogeneous and more challenging heterogeneous settings.
Blogger's Review: The introduction of ProMoE-FL offers an innovative solution for multimodal federated learning, particularly in handling missing modalities. By constructing a prototype bank and employing conditional routing, it significantly enhances the robustness of feature synthesis, showing great potential for broad applications. Its superior performance across multiple datasets showcases the effectiveness and forward-looking nature of this approach.