A one-size-fits-all approach likely isn’t the best strategy when designing artificial intelligence systems that assist users in disease diagnosis. A new study by researchers at MIT and elsewhere found that, while AI assistance generally improved the accuracy of non-experts and clinicians in diagnosing skin diseases, AI explainability methods had different impacts depending on the users’ knowledge level. Explainable AI methods help users know when to trust a model’s predictions by describing or validating the model’s decision-making. For instance, a model might use a heat map to highlight image regions that were most important in its diagnosis or a large language model (LLM) to explain the prediction in plain language. The researchers explored the effects and potential benefits of these explainable AI tools on primary care physicians and non-experts in dermatological disease detection. They found that non-experts’ diagnostic accuracy improved, but it was largely due to deference to the AI system. Blogger's Review: This study highlights the importance of designing AI systems with users in mind, particularly considering the level of expertise of non-experts and clinicians. The findings also underscore the need for developing explainability methods that promote critical thinking rather than overreliance on the model. By doing so, we can create more effective AI systems that improve healthcare outcomes while minimizing the risks of automation bias.