Rxivist logo

The Impact of Digital Histopathology Batch Effect on Deep Learning Model Accuracy and Bias

By Frederick Matthew Howard, James Dolezal, Sara Kochanny, Jefree Schulte, Heather Chen, Lara Heij, Dezheng Huo, Rita Nanda, Olufunmilayo I. Olopade, Jakob Kather, Robert Grossman, Alexander T Pearson

Posted 04 Dec 2020
bioRxiv DOI: 10.1101/2020.12.03.410845

The Cancer Genome Atlas (TCGA) is one of the largest biorepositories of digital histology. Deep learning (DL) models have been trained on TCGA to predict numerous features directly from histology, including survival, gene expression patterns, and driver mutations. However, we demonstrate that these features vary substantially across tissue submitting sites in TCGA for over 3,000 patients with six cancer subtypes. We demonstrate that staining differences between submitting sites exist and can easily be identified by DL models. We quantify the digital image characteristics constituting this histologic batch effect. We demonstrate that DL models can accurately predict submitting site with an area under the receiver operating characteristic curve of 0.850 or higher despite commonly used stain normalization and augmentation methods. To correct for biased estimates of accuracy, we propose a mixed integer quadratic programming method to separate data into cross folds to abrogate this bias. As a demonstration, we show that patient ethnicity can be inferred from histology due to site-level staining patterns; this must be accounted for to ensure equitable applicability of deep learning models.

Download data

  • Downloaded 344 times
  • Download rankings, all-time:
    • Site-wide: 76,766
    • In bioinformatics: 7,103
  • Year to date:
    • Site-wide: 15,038
  • Since beginning of last month:
    • Site-wide: 11,410

Altmetric data

Downloads over time

Distribution of downloads per paper, site-wide


Sign up for the Rxivist weekly newsletter! (Click here for more details.)