NCKU CSIE & Harvard Reveal AI Diagnostic Bias, Featured as a Cover Article in Cell Reports Medicine-國立成功大學永續發展SDGs

Artificial intelligence (AI) has been rapidly integrated into the medical field in recent years and has become an important tool assisting physicians in interpreting pathological specimens. However, can AI systems—often regarded as “objective and rational”—truly treat all patients equally? A collaborative study by the Department of Computer Science and Information Engineering at National Cheng Kung University (NCKU) and Harvard Medical School reveals hidden risks of diagnostic bias in AI systems. The paper, titled “Contrastive Learning Enhances Fairness in Pathology Artificial Intelligence Systems,” was published in the leading international medical journal Cell Reports Medicine and selected as a cover article.

In medical applications—particularly in cancer pathology image analysis—AI has significantly improved diagnostic efficiency and accuracy. However, the NCKU–Harvard research team found that many pathology AI models currently widely used in clinical and research settings may, without users’ awareness, produce systematic diagnostic biases. These biases can affect the quality of diagnoses received by different patient groups, such as Western and East Asian populations.

The research team collected and analyzed large-scale pathology image datasets spanning 23 types of cancer, including breast and lung cancers, and systematically evaluated multiple commonly used pathology AI models. The results showed that although these models are designed to make disease judgments based solely on pathological features, in practice they may inadvertently infer latent cues related to patients’ sex, age, or ethnic background from images. By incorporating such non-disease-related information into decision-making, the models can produce significant disparities in diagnostic accuracy across different population groups.

Overall, the analysis found that approximately one-third of AI-based clinical diagnoses exhibited statistically significant performance differences between groups. This indicates that even when a model demonstrates strong average accuracy, certain populations may still bear a higher risk of misdiagnosis. These findings directly challenge the long-held assumption that medical AI systems are inherently objective and fair.

Further investigation revealed that the sources of diagnostic bias are not limited to imbalanced group representation in training data. They are also associated with differences in disease prevalence across populations and with AI models developing “shortcut learning” behaviors—over-reliance on image features that are highly correlated with group identity but not intrinsically related to disease. Although such features may have no direct pathological relevance, their strong association with certain populations can unintentionally amplify diagnostic inequities.

Addressing this phenomenon, Distinguished Professor Jung-Hsien Chiang (蔣榮先) of NCKU’s Department of Computer Science and Information Engineering explained that AI systems may sometimes “take shortcuts” during learning. Instead of focusing on truly disease-relevant features, they may latch onto image characteristics that frequently co-occur with certain populations. While these features do not necessarily indicate illness, the AI may mistakenly treat them as diagnostic signals, increasing the risk of unfair outcomes for specific groups.

To tackle this challenge, the NCKU research team—led by Distinguished Professor Chiang and joined by doctoral students Pei-Jen Tsai (蔡沛蓁) and Fang-Yi Su (蘇芳毅)—focused on the development of Trustworthy AI. They proposed a novel framework, FAIR-Path (Fairness-Aware Representation Learning for Pathology), which uses carefully designed training strategies to guide models to focus on pathology features that are highly relevant to disease while reducing reliance on group-related image cues. Professor Chiang noted that the team’s core research theme is Trustworthy AI (TW-AI), a term that also carries the dual meaning of “Taiwan AI,” underscoring the team’s commitment to developing AI technologies that are both trustworthy and internationally competitive.

During training, FAIR-Path continually reinforces a key principle to the model: if two patients have the same disease, their images should be considered similar even if they come from different population groups; conversely, if the diseases differ, images should not be conflated even when the patients share similar backgrounds. This process is analogous to training medical students to focus on tumor structures and cellular changes rather than making assumptions based on a patient’s background. As a result, the AI is less likely to learn biased shortcuts and more likely to make fair, disease-centered judgments.

Experimental results demonstrate that after incorporating FAIR-Path, diagnostic consistency across different population groups improved by an average of approximately 88%, without sacrificing overall diagnostic performance. The framework significantly enhanced both fairness and trustworthiness, highlighting its strong potential for real-world clinical application.

This cross-national research on fairness and reliability in medical AI was co-led by Distinguished Professor Chiang and a Harvard Medical School professor as corresponding authors. The study was also featured in a dedicated report by Harvard Medical School, reflecting strong international academic recognition. The research team emphasized that this work not only provides a critical warning for the clinical deployment of medical AI but also offers a concrete and feasible technical solution, laying an essential foundation for the future development of fairer and more trustworthy intelligent healthcare systems.