Automatic Detection of Steatosis in Ultrasound Images with Comparative Visual Labeling

A common difficulty in computer-assisted diagnosis is acquiring accurate and representative labeled data, required to train, test and monitor models. Concerning liver steatosis detection in ultrasound (US) images, labeling images with human annotators can be error-prone because of subjectivity and decision boundary biases. To overcome these limits, we propose comparative visual labeling (CVL), where an annotator labels the relative degree of a pathology in image pairs, that is combined with a RankNet to give per-image diagnostic scores. In a multi-annotator evaluation on a public steatosis dataset, CVL+RankNet significantly improves label quality compared to conventional single-image visual labeling (SVL) (0.97 versus 0.87 F1-score respectively, 95% CI significance). This is the first application of CVL for diagnostic medical image labeling, and it may stimulate more research for other diagnostic labeling tasks. We also show that Deep Learning (DL) models trained with CVL+RankNet or histopathology labels attain similar performance.