External Validation of a Deep Learning Model for Predicting Mammographic Breast Density in Routine Clinical Practice

Rationale and Objectives Federal legislation requires patient notification of dense mammographic breast tissue because increased density is a marker of breast cancer risk and can limit the sensitivity of mammography. As previously described, we clinically implemented our deep learning model at the academic breast imaging practice where the model was developed with high clinical acceptance. Our objective was to externally validate our deep learning model on radiologist breast density assessments in a community breast imaging practice. Materials and Methods Our deep learning model was implemented at a dedicated breast imaging practice staffed by both academic and community breast imaging radiologists in October 2018. Deep learning model assessment of mammographic breast density was presented to the radiologist during routine clinical practice at the time of mammogram interpretation. We identified 2174 consecutive screening mammograms after implementation of the deep learning model. Radiologist agreement with the model's assessment was measured and compared across radiologist groups. Results Both academic and community radiologists had high clinical acceptance of the deep learning model's density prediction, with 94.9% (academic) and 90.7% (community) acceptance for dense versus nondense categories (p < 0.001). The proportion of mammograms assessed as dense by all radiologists decreased from 47.0% before deep learning model implementation to 41.0% after deep learning model implementation (p < 0.001). Conclusion Our deep learning model had a high clinical acceptance rate among both academic and community radiologists and reduced the proportion of mammograms assessed as dense. This is an important step to validating our deep learning model prior to potential widespread implementation. Federal legislation requires patient notification of dense mammographic breast tissue because increased density is a marker of breast cancer risk and can limit the sensitivity of mammography. As previously described, we clinically implemented our deep learning model at the academic breast imaging practice where the model was developed with high clinical acceptance. Our objective was to externally validate our deep learning model on radiologist breast density assessments in a community breast imaging practice. Our deep learning model was implemented at a dedicated breast imaging practice staffed by both academic and community breast imaging radiologists in October 2018. Deep learning model assessment of mammographic breast density was presented to the radiologist during routine clinical practice at the time of mammogram interpretation. We identified 2174 consecutive screening mammograms after implementation of the deep learning model. Radiologist agreement with the model's assessment was measured and compared across radiologist groups. Both academic and community radiologists had high clinical acceptance of the deep learning model's density prediction, with 94.9% (academic) and 90.7% (community) acceptance for dense versus nondense categories (p < 0.001). The proportion of mammograms assessed as dense by all radiologists decreased from 47.0% before deep learning model implementation to 41.0% after deep learning model implementation (p < 0.001). Our deep learning model had a high clinical acceptance rate among both academic and community radiologists and reduced the proportion of mammograms assessed as dense. This is an important step to validating our deep learning model prior to potential widespread implementation.