CheXMed: A multimodal learning algorithm for pneumonia detection in the elderly

Pneumonia can be a deadly illness for particular populations, one of which is older adults. While studies have successfully trained artificial intelligent assisted diagnostic tools to detect pneumonia using chest X-ray images, they were targeted to the general population without stratification on age groups. This study (a) investigated the performance disparities between geriatric and younger patients when using chest X-ray images to detect pneumonia, and (b) developed and tested a multimodal model called CheXMed that incorporates clinical notes together with image data to improve pneumonia detection performance for older people. Accuracy, precision, recall, and F1-score were used for model performance evaluation. CheXMed outperforms baseline models on all evaluation metrics. The accuracy, precision, recall, and F1-score are 0.746, 0.746, 0.740, 0.743 for CheXMed, 0.645, 0.680, 0.535, 0.599 for CheXNet, 0.623, 0.655, 0.521, 0.580 for DenseNet121, and 0.610, 0.617, 0.543, 0.577 for ResNet18.