A deep learning-based system for identifying differentiation status and delineating the margins of early gastric cancer in magnifying narrow-band imaging endoscopy

Abstract Background Accurate identification of the differentiation status and margins for early gastric cancer (EGC) is critical for determining the surgical strategy and achieving curative resection in EGC patients. The aim of this study was to develop a real-time system to accurately identify differentiation status and delineate the margins of EGC on magnifying narrow-band imaging (ME-NBI) endoscopy. Methods 2217 images from 145 EGC patients and 1870 images from 139 EGC patients were retrospectively collected to train and test the first convolutional neural network (CNN1) to identify EGC differentiation status. The performance of CNN1 was then compared with that of experts using 882 images from 58 EGC patients. Finally, 928 images from 132 EGC patients and 742 images from 87 EGC patients were used to train and test CNN2 to delineate the EGC margins. Results The system correctly predicted the differentiation status of EGCs with an accuracy of 83.3 % (95 % confidence interval [CI] 81.5 % – 84.9 %) in the testing dataset. In the man – machine contest, CNN1 performed significantly better than the five experts (86.2 %, 95 %CI 75.1 % – 92.8 % vs. 69.7 %, 95 %CI 64.1 % – 74.7 %). For delineating EGC margins, the system achieved an accuracy of 82.7 % (95 %CI 78.6 % – 86.1 %) in differentiated EGC and 88.1 % (95 %CI 84.2 % – 91.1 %) in undifferentiated EGC under an overlap ratio of 0.80. In unprocessed EGC videos, the system achieved real-time diagnosis of EGC differentiation status and EGC margin delineation in ME-NBI endoscopy. Conclusion We developed a deep learning-based system to accurately identify differentiation status and delineate the margins of EGC in ME-NBI endoscopy. This system achieved superior performance when compared with experts and was successfully tested in real EGC videos.