Deep learning models achieve state-of-the-art performance in image classification but face deployment challenges due to computational costs and energy demands. We propose a lightweight training strategy that adapts the normalization layers of the model to the new domain and decouples feature extraction from classifier optimization, reducing overhead by precomputing features only once.
A redesigned classifier head with margin-based weighted loss further minimizes ambiguity without the need for end-to-end backpropagation. Evaluated across four CNN architectures (ResNet18, ResNet50, MobileNet, DenseNet121), three Transformer models (ViT, Swin, and DeiT), and three medical datasets (Brain Cancer MRI, BreakHis, and PatchCamelyon), our approach significantly reduces the required training time with only a marginal accuracy trade-off, often matching or surpassing baseline performance. This efficiency translates to reducing CO2 emissions by orders of magnitude, offering a practical and environmentally sustainable solution for resource-constrained clinical or prototyping environments.
Blogger's Review: This study highlights an effective approach to transfer learning by decoupling feature extraction from classifier optimization, demonstrating how training time and resource consumption can be significantly reduced while maintaining model performance. Its implications are particularly valuable in resource-limited medical environments, showcasing its practical applications.