Application of Artificial Intelligence to Gastroenterology and Hepatology

Since 2010, substantial progress has been made in artificial intelligence (AI) and its application to medicine. AI is explored in gastroenterology for endoscopic analysis of lesions, in detection of cancer, and to facilitate the analysis of inflammatory lesions or gastrointestinal bleeding during wireless capsule endoscopy. AI is also tested to assess liver fibrosis and to differentiate patients with pancreatic cancer from those with pancreatitis. AI might also be used to establish prognoses of patients or predict their response to treatments, based on multiple factors. We review the ways in which AI may help physicians make a diagnosis or establish a prognosis and discuss its limitations, knowing that further randomized controlled studies will be required before the approval of AI techniques by the health authorities. Since 2010, substantial progress has been made in artificial intelligence (AI) and its application to medicine. AI is explored in gastroenterology for endoscopic analysis of lesions, in detection of cancer, and to facilitate the analysis of inflammatory lesions or gastrointestinal bleeding during wireless capsule endoscopy. AI is also tested to assess liver fibrosis and to differentiate patients with pancreatic cancer from those with pancreatitis. AI might also be used to establish prognoses of patients or predict their response to treatments, based on multiple factors. We review the ways in which AI may help physicians make a diagnosis or establish a prognosis and discuss its limitations, knowing that further randomized controlled studies will be required before the approval of AI techniques by the health authorities. There is no single definition of artificial intelligence (AI), but the concept involves computer programs that perform functions that we associate with human intelligence, such as learning and problem solving.1Russell S. Norvig P. Artificial Intelligence: A Modern Approach, Global Edition.3rd ed. Pearson, London, UK2016Google Scholar,2Colom R. Karama S. Jung R.E. et al.Human intelligence and brain networks.Dialogues Clin Neurosci. 2010; 12: 489-501Crossref PubMed Google Scholar AI, machine learning (ML), and deep learning (DL) are overlapping disciplines (Figure 1). ML is a vast domain that involves computer science and statistics in which a machine performs repeated iterations of models progressively improving performance of a specific task. It produces algorithms to analyze data and to learn descriptive and predictive models. Data are mostly in the form of tables with objects or individuals as rows and variables, either numerical or categorical, as columns. ML is roughly divided into supervised and unsupervised methods. Unsupervised learning occurs when the purpose is to identify groups within data according to commonalities, with no a priori knowledge of the number of groups or their significance. Supervised learning occurs when training data contain individuals represented as input–output pairs. Input comprises individual descriptors, whereas output comprises outcomes of interest to be predicted—either a class for classification tasks or a numerical value for regression tasks. The supervised ML algorithm then learns predictive models that subsequently allow mapping new inputs to outputs.3Shalev-Shwartz S. Ben-David S. Understanding Machine Learning: From Theory to Algorithms. Cambridge University Press, New York2014Crossref Scopus (592) Google Scholar Artificial neural networks (ANN) are supervised ML models inspired by the neuroanatomy of brain. Each neuron is a computing unit and all neurons are connected to each other to build a network. Signals travel from the first (input), to the last (output) layer, possibly after going through multiple hidden layers (Figure 2). Training an ANN consists of dividing the data into a training set, which helps to define the architecture of the network and to find out the various weights between the nodes, and then a test set to assess the capability of the ANN to predict the desired output. During training, weights of interneuron connections are adjusted to optimize classification. The competition for more performance has led to a progressive complexity of neural network architectures resulting in the concept of DL.4LeCun Y. Bengio Y. Hinton G. Deep learning.Nature. 2015; 521: 436-444Crossref PubMed Scopus (15892) Google Scholar Deep neural network (DNN) models are characterized by the application of several consecutive filters that allow the automatic detection of relevant features of input data. For this reason, DNNs are considered as capable of learning data representation while including this learning in the global learning of the classification task. A variety of DNN architectures are included in DL-based methods.5Goodfellow I. Bengio Y. Courville A. Deep Learning. The MIT Press, Cambridge, MA2016Google Scholar However, the good performance obtained requires a huge amount of labeled training data. Researchers have addressed this issue by combining DL with reinforcement learning principles.6Mahmud M. Kaiser M.S. Hussain A. et al.Applications of deep learning and reinforcement learning to biological data.IEEE Trans Neural Netw Learn Syst. 2018; 29: 2063-2079Crossref PubMed Scopus (0) Google Scholar The limits to these techniques are overfitting and lack of explainability. The models obtained by DL often perform much better than any other at fitting the data, however, they are intrinsically dependent on the training dataset. If the training population does not include enough diversity, or contains an unidentified bias, results may not be generalizable to real-life populations, leading to problems in model validation. Moreover, DNNs, like ANNs, provide black-box models lacking explainability. Recent studies are oriented towards improving explainability of DNN models, as it is a prerequisite for their acceptability in many fields, particularly in the biomedical applications.7Montavon G. Samek W. Müller K.-R. Methods for interpreting and understanding deep neural networks.Digital Signal Proc. 2018; 73: 1-15Crossref Scopus (0) Google Scholar,8Japkowicz N. Shah M. Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press, New York2011Crossref Google Scholar There have been reviews on the use of AI in gastroenterology, but they focused mainly on AI assisted-endoscopy.9Ahmad O.F. Soares A.S. Mazomenos E. et al.Artificial intelligence and computer-aided diagnosis in colonoscopy: current evidence and future directions.Lancet Gastroenterol Hepatol. 2019; 4: 71-80Abstract Full Text Full Text PDF PubMed Google Scholar, 10Ruffle J.K. Farmer A.D. Aziz Q. Artificial intelligence-assisted gastroenterology— promises and pitfalls.Am J Gastroenterol. 2019; 114: 422-428Crossref PubMed Scopus (3) Google Scholar, 11Iakovidis D.K. Koulaouzidis A. Software for enhanced video capsule endoscopy: challenges for essential progress.Nat Rev Gastroenterol Hepatol. 2015; 12: 172-186Crossref PubMed Scopus (98) Google Scholar We provide an overview of important studies assessing the value of AI in helping physicians make a diagnosis or establish a prognosis in the main fields of gastroenterology and hepatology (Supplementary Table 1 and Supplementary Figures 1 and 2). Most studies use 1 dataset to train the ML process and a second dataset to test its performance. Some studies use common evaluation techniques, such as cross-validation and leave 1 out.8Japkowicz N. Shah M. Evaluating Learning Algorithms: A Classification Perspective. Cambridge University Press, New York2011Crossref Google Scholar To increase the size of the dataset, some studies use image-applied data augmentation by a random resizing and cropping of the frame, followed by a random flipping along either axis. Datasets can include images of negative (normal) results and positive (pathologic) results. Fifty-three studies have used AI to detect malignant and premalignant intestinal lesions (Table 1). Most of these (n = 48) focused on endoscopy, 3 studies used clinical and biological data extracted from electronic medical records (mainly demographics, cardiovascular comorbidities, concomitant medication, digestive symptoms, and complete blood count), 1 study was based on serum tumor markers, and 1 study used data from gut microbiota. Twenty-seven studies were dedicated to improving diagnostic accuracy in case of colorectal polyps or cancer.12Häfner M. Brunauer L. Payer H. et al.Computer-aided classification of zoom-endoscopical images using Fourier filters.IEEE Trans Inf Technol Biomed. 2010; 14: 958-970Crossref PubMed Scopus (0) Google Scholar, 13Ribeiro E. Uhl A. Wimmer G. et al.Exploring deep learning and transfer learning for colonic polyp classification.Comput Math Methods Med. 2016; 2016 (Available at:)https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5101370/Date accessed: September 14, 2018Crossref PubMed Scopus (1) Google Scholar, 14Mesejo P. Pizarro D. Abergel A. et al.Computer-aided classification of gastrointestinal lesions in regular colonoscopy.IEEE Trans Med Imaging. 2016; 35: 2051-2063Crossref PubMed Scopus (30) Google Scholar, 15Bernal J. Tajkbaksh N. Sanchez F.J. et al.Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 Endoscopic Vision Challenge.IEEE Trans Med Imaging. 2017; 36: 1231-1249Crossref PubMed Scopus (77) Google Scholar, 16Byrne M.F. Chapados N. Soudan F. et al.Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.Gut. 2019; 68: 94-100Crossref PubMed Scopus (88) Google Scholar, 17Komeda Y. Handa H. Watanabe T. et al.Computer-aided diagnosis based on convolutional neural network system for colorectal polyp classification: preliminary experience.Oncology. 2017; 93: 30-34Crossref PubMed Scopus (34) Google Scholar, 18Billah M. Waheed S. Rahman M.M. An automatic gastrointestinal polyp detection system in video endoscopy using fusion of color wavelet and convolutional neural network features.Int J Biomed Imaging. 2017; 2017: 9545920Crossref PubMed Scopus (19) Google Scholar, 19Misawa M. Kudo S.-E. Mori Y. et al.Artificial intelligence-assisted polyp detection for colonoscopy: initial experience.Gastroenterology. 2018; 154: 2027-2029.e3Abstract Full Text Full Text PDF PubMed Scopus (49) Google Scholar, 20Chen P.-J. Lin M.-C. Lai M.-J. et al.Accurate classification of diminutive colorectal polyps using computer-aided analysis.Gastroenterology. 2018; 154: 568-575Abstract Full Text Full Text PDF PubMed Scopus (64) Google Scholar, 21Urban G. Tripathi P. Alkayali T. et al.Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy.Gastroenterology. 2018; 155: 1069-1078.e8Abstract Full Text Full Text PDF PubMed Scopus (71) Google Scholar, 22Renner J. Phlipsen H. Haller B. et al.Optical classification of neoplastic colorectal polyps—a computer-assisted approach (the COACH study).Scand J Gastroenterol. 2018; 53: 1100-1106Crossref PubMed Scopus (1) Google Scholar, 23Wang P. Xiao X. Brown J.R.G. et al.Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy.Nat Biomed Eng. 2018; 2: 741Crossref PubMed Scopus (38) Google Scholar, 24Min M. Su S. He W. et al.Computer-aided diagnosis of colorectal polyps using linked color imaging colonoscopy to predict histology.Sci Rep. 2019; 9: 2881Crossref PubMed Scopus (0) Google Scholar, 25Mori Y. Kudo S.-E. Misawa M. et al.Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study.Ann Intern Med. 2018; 169: 357-366Crossref PubMed Scopus (50) Google Scholar, 26Misawa M. Kudo S. Mori Y. et al.Accuracy of computer-aided diagnosis based on narrow-band imaging endocytoscopy for diagnosing colorectal lesions: comparison with experts.Int J Comp Assist Radiol Surg. 2017; 12: 757-766Crossref PubMed Scopus (0) Google Scholar, 27Romain O, Histace A, Silva J, et al. Towards a multimodal wireless video capsule for detection of colonic polyps as prevention of colorectal cancer. Proceedings of the 13th IEEE International Conference on BioInformatics and BioEngineering; November 10–13, 2013; Chania, Greece; pp 1–6.Google Scholar, 28David E, Boia R, Malaescu A, et al. Automatic colon polyp detection in endoscopic capsule images. Proceedings of the International Symposium on Signals, Circuits and Systems ISSCS2013; July 11–12, 2013; Iasi, Romania; pp 1–4.Google Scholar, 29Mamonov A.V. Figueiredo I.N. Figueiredo P.N. et al.Automated polyp detection in colon capsule endoscopy.IEEE Trans Med Imaging. 2014; 33: 1488-1502Crossref PubMed Scopus (0) Google Scholar, 30Blanes-Vidal V. Baatrup G. Nadimi E.S. Addressing priority challenges in the detection and assessment of colorectal polyps from capsule endoscopy and colonoscopy in colorectal cancer screening using machine learning.Acta Oncol. 2019; 58: S29-S36Crossref PubMed Scopus (1) Google Scholar, 31Ito N. Kawahira H. Nakashima H. et al.Endoscopic diagnostic support system for cT1b colorectal cancer using deep learning.Oncology. 2019; 96: 44-50Crossref PubMed Scopus (0) Google Scholar, 32Ştefănescu D. Streba C. Cârţână E.T. et al.Computer aided diagnosis for confocal laser endomicroscopy in advanced colorectal adenocarcinoma.PLoS One. 2016; 11e0154863Crossref PubMed Scopus (17) Google Scholar, 33Takeda K. Kudo S.-E. Mori Y. et al.Accuracy of diagnosing invasive colorectal cancer using computer-aided endocytoscopy.Endoscopy. 2017; 49: 798-802Crossref PubMed Scopus (22) Google Scholar, 34Kop R. Hoogendoorn M. Teije A.T. et al.Predictive modeling of colorectal cancer using a dedicated pre-processing pipeline on routine electronic medical records.Comput Biol Med. 2016; 76: 30-38Crossref PubMed Google Scholar, 35Hoogendoorn M. Szolovits P. Moons L.M.G. et al.Utilizing uncoded consultation notes from electronic medical records for predictive modeling of colorectal cancer.Artif Intell Med. 2016; 69: 53-61Crossref PubMed Google Scholar, 36Kinar Y. Akiva P. Choman E. et al.Performance analysis of a machine learning flagging system used to identify a group of individuals at a high risk for colorectal cancer.PLoS One. 2017; 12e0171759Crossref PubMed Scopus (7) Google Scholar, 37Zhang B. Liang X.L. Gao H.Y. et al.Models of logistic regression analysis, support vector machine, and back-propagation neural network based on serum tumor markers in colorectal cancer diagnosis.Genet Mol Res. 2016; 15Google Scholar, 38Ai L. Tian H. Chen Z. et al.Systematic evaluation of supervised classifiers for fecal microbiota-based prediction of colorectal cancer.Oncotarget. 2017; 8: 9546-9556Crossref PubMed Scopus (22) Google Scholar Nineteen studies focused on the diagnosis of premalignant or malignant lesions of the upper gastrointestinal tract,39van der Sommen F. Zinger S. Curvers W.L. et al.Computer-aided detection of early neoplastic lesions in Barrett’s esophagus.Endoscopy. 2016; 48: 617-624Crossref PubMed Scopus (42) Google Scholar, 40Swager A.-F. Sommen F van der Klomp S.R. et al.Computer-aided detection of early Barrett’s neoplasia using volumetric laser endomicroscopy.Gastrointest Endosc. 2017; 86: 839-846Abstract Full Text Full Text PDF PubMed Scopus (24) Google Scholar, 41Riaz F. Ribeiro M.-D. Pimentel-Nunes P. et al.Integral scale histogram local binary patterns for classification of narrow-band gastroenterology images.Conf Proc IEEE Eng Med Biol Soc. 2013; 2013: 3714-3717PubMed Google Scholar, 42Horie Y. Yoshio T. Aoyama K. et al.The diagnostic outcomes of esophageal cancer by artificial intelligence using convolutional neural networks.Gastrointest Endosc. 2019; 89: 25-32Abstract Full Text Full Text PDF PubMed Scopus (0) Google Scholar, 43Kumagai Y. Takubo K. Kawada K. et al.Diagnosis using deep-learning artificial intelligence based on the endocytoscopic observation of the esophagus.Esophagus. 2019; 16: 180-187Crossref PubMed Scopus (2) Google Scholar, 44Zhao Y.-Y. Xue D.-X. Wang Y.-L. et al.Computer-assisted diagnosis of early esophageal squamous cell carcinoma using narrow-band imaging magnifying endoscopy.Endoscopy. 2019; 51: 333-341Crossref PubMed Scopus (8) Google Scholar, 45Shichijo S. Nomura S. Aoyama K. et al.Application of convolutional neural networks in the diagnosis of Helicobacter pylori infection based on endoscopic images.EBioMedicine. 2017; 25: 106-111Abstract Full Text Full Text PDF PubMed Scopus (58) Google Scholar, 46Itoh T. Kawahira H. Nakashima H. et al.Deep learning analyzes Helicobacter pylori infection by upper gastrointestinal endoscopy images.Endosc Int Open. 2018; 6: E139-E144Crossref PubMed Google Scholar, 47Hirasawa T. Aoyama K. Tanimoto T. et al.Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images.Gastric Cancer. 2018; 21: 653-660Crossref PubMed Scopus (75) Google Scholar, 48Ogawa R. Nishikawa J. Hideura E. et al.Objective assessment of the utility of chromoendoscopy with a support vector machine.J Gastrointest Cancer. 2019; 50: 386-391Crossref PubMed Scopus (0) Google Scholar, 49Ali H. Yasmin M. Sharif M. et al.Computer assisted gastric abnormalities detection using hybrid texture descriptors for chromoendoscopy images.Comput Methods Prog Biomed. 2018; 157: 39-47Crossref PubMed Scopus (5) Google Scholar, 50Sakai Y. Takemoto S. Hori K. et al.Automatic detection of early gastric cancer in endoscopic images using a transferring convolutional neural network.Conf Proc IEEE Eng Med Biol Soc. 2018; 2018: 4138-4141PubMed Google Scholar, 51Kanesaka T. Lee T.-C. Uedo N. et al.Computer-aided diagnosis for identifying and delineating early gastric cancers in magnifying narrow-band imaging.Gastrointest Endosc. 2018; 87: 1339-1344Abstract Full Text Full Text PDF PubMed Scopus (20) Google Scholar, 52Wu L. Zhou W. Wan X. et al.A deep neural network improves endoscopic detection of early gastric cancer without blind spots.Endoscopy. 2019; (Available at:)http://www.thieme-connect.de/DOI/DOI?10.1055/a-0855-3532Date accessed: April 30, 2019Google Scholar, 53Lee J.H. Kim Y.J. Kim Y.W. et al.Spotting malignancies from gastric endoscopic images using deep learning.Surg Endosc. 2019; 33: 3790-3797Crossref PubMed Scopus (0) Google Scholar, 54Riaz F. Silva F.B. Ribeiro M.D. et al.Invariant Gabor texture descriptors for classification of gastroenterology images.IEEE Trans Biomed Eng. 2012; 59: 2893-2904Crossref PubMed Scopus (0) Google Scholar, 55Liu D.-Y. Gan T. Rao N.-N. et al.Identification of lesion images from gastrointestinal endoscope based on feature extraction of combinational methods with and without learning process.Med Image Anal. 2016; 32: 281-294Crossref PubMed Scopus (0) Google Scholar, 56Kubota K. Kuroda J. Yoshida M. et al.Medical image analysis: computer-aided diagnosis of gastric cancer invasion on endoscopic images.Surg Endosc. 2012; 26: 1485-1489Crossref PubMed Scopus (6) Google Scholar, 57Zhu Y. Wang Q.-C. Xu M.-D. et al.Application of convolutional neural network in the diagnosis of the invasion depth of gastric cancer based on conventional endoscopy.Gastrointest Endosc. 2019; 89: 806-815.e1Abstract Full Text Full Text PDF PubMed Scopus (22) Google Scholar only 4 studies were limited to the small bowel,58Constantinescu A.F. Ionescu M. Iov V.-F. et al.A computer-aided diagnostic system for intestinal polyps identified by wireless capsule endoscopy.Rom J Morphol Embryol. 2016; 57: 979-984PubMed Google Scholar, 59Li B. Meng M.Q.-H. Lau J.Y.W. Computer-aided small bowel tumor detection for capsule endoscopy.Artif Intell Med. 2011; 52: 11-16Crossref PubMed Scopus (0) Google Scholar, 60Faghih Dinevari V. Karimian Khosroshahi G. Zolfy Lighvan M. Singular value decomposition based features for automatic tumor detection in wireless capsule endoscopy images.Appl Bionics Biomech. 2016; 2016: 3678913Crossref PubMed Scopus (1) Google Scholar, 61Liu G. Yan G. Kuang S. et al.Detection of small bowel tumor based on multi-scale curvelet analysis and fractal technology in capsule endoscopy.Comput Biol Med. 2016; 70: 131-138Crossref PubMed Google Scholar and 3 studies assessed the entire digestive tract.62Li B. Meng M.Q.H. Xu L. A comparative study of shape features for polyp detection in wireless capsule endoscopy images.Conf Proc IEEE Eng Med Biol Soc. 2009; 2009: 3731-3734PubMed Google Scholar, 63Yuan Y. Meng M.Q.-H. Deep learning for polyp recognition in wireless capsule endoscopy images.Med Phys. 2017; 44: 1379-1389Crossref PubMed Scopus (41) Google Scholar, 64Baopu Li Meng M.Q.-H. Tumor recognition in wireless capsule endoscopy images using textural features and SVM-based feature selection.IEEE Trans Inform Technol Biomed. 2012; 16: 323-329Crossref PubMed Scopus (75) Google Scholar Twenty-four studies used specific validation techniques—mainly k-fold cross-validation. For studies focusing on endoscopy, the size of training and test datasets varied widely across studies. Performance results were also heterogeneous from one study to another, but most of the presented algorithms reached an accuracy of >80%.Table 1Use of Artificial Intelligence in Identification of Patients With Intestinal Malignancies or Premalignant LesionsLesionsDiagnostic or predictive modalityAI classifierAI validation methodsNo. of images/casesaNumber of images (frames or videos) for studies analyzing endoscopy. Number of cases (patients) for studies analyzing electronic medical records, microbiota, serum markers. in training dataset (negative/positive)bThe number of negative and positive data is provided if applicable.No. of images/casesaNumber of images (frames or videos) for studies analyzing endoscopy. Number of cases (patients) for studies analyzing electronic medical records, microbiota, serum markers. in test dataset (negative/positive)bThe number of negative and positive data is provided if applicable.Best average results, %ReferenceAccuracySensitivity/SpecificityColon and rectum PolypsHigh-magnification colonoscopyRegularized discriminant analysis or SVMLOO484 (198 non-adenomas/286 adenomas)96.997.2/96.012Häfner M. Brunauer L. Payer H. et al.Computer-aided classification of zoom-endoscopical images using Fourier filters.IEEE Trans Inf Technol Biomed. 2010; 14: 958-970Crossref PubMed Scopus (0) Google Scholar PolypsColonoscopy (CE)CNNLOO100 (75/25)cAfter data augmentation.2500 (1,875/625)93.6NA13Ribeiro E. Uhl A. Wimmer G. et al.Exploring deep learning and transfer learning for colonic polyp classification.Comput Math Methods Med. 2016; 2016 (Available at:)https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5101370/Date accessed: September 14, 2018Crossref PubMed Scopus (1) Google Scholar PolypsColonoscopy (WL or NBI)RF, random subspace, or SVMLOO76 videos (15 serrated/21 hyperplastic/40 adenomas)82.572.7/85.914Mesejo P. Pizarro D. Abergel A. et al.Computer-aided classification of gastrointestinal lesions in regular colonoscopy.IEEE Trans Med Imaging. 2016; 35: 2051-2063Crossref PubMed Scopus (30) Google Scholar PolypsColonoscopySeveral CNNNot applicable612 frames + 20 videos (10/10)192 frames + 18 videos (9/9)Several methods compared15Bernal J. Tajkbaksh N. Sanchez F.J. et al.Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 Endoscopic Vision Challenge.IEEE Trans Med Imaging. 2017; 36: 1231-1249Crossref PubMed Scopus (77) Google Scholar PolypsColonoscopy (NBI)CNNRandom subsampling60,089cAfter data augmentation. (223 videos; 29%type 1 and 53%type 2 based on NBI International Colorectal Endoscopic; 18% no polyp)125 (51 hyperplastic/74 adenomas94.098.0/83.016Byrne M.F. Chapados N. Soudan F. et al.Real-time differentiation of adenomatous and hyperplastic diminutive colorectal polyps during analysis of unaltered videos of standard colonoscopy using a deep learning model.Gut. 2019; 68: 94-100Crossref PubMed Scopus (88) Google Scholar PolypsColonoscopyCNN10-fold cross-validation1200 (600/600)1070.083.3/50.017Komeda Y. Handa H. Watanabe T. et al.Computer-aided diagnosis based on convolutional neural network system for colorectal polyp classification: preliminary experience.Oncology. 2017; 93: 30-34Crossref PubMed Scopus (34) Google Scholar PolypsColonoscopySVM—d—, no specific validation technique was used (or missing data).100 videos split into training and test datasets98.798.8/98.518Billah M. Waheed S. Rahman M.M. An automatic gastrointestinal polyp detection system in video endoscopy using fusion of color wavelet and convolutional neural network features.Int J Biomed Imaging. 2017; 2017: 9545920Crossref PubMed Scopus (19) Google Scholar PolypsColonoscopyCNN—196,631 (133,496/63,135)76.590.0/63.319Misawa M. Kudo S.-E. Mori Y. et al.Artificial intelligence-assisted polyp detection for colonoscopy: initial experience.Gastroenterology. 2018; 154: 2027-2029.e3Abstract Full Text Full Text PDF PubMed Scopus (49) Google Scholar411 videos (306/105)135 videos (85/50) PolypsHigh-magnification colonoscopy (NBI)CNN—2,157 (681 hyperplastic/1476 adenomas)284 (96 hyperplastic/188 adenomas)90.196.3/78.120Chen P.-J. Lin M.-C. Lai M.-J. et al.Accurate classification of diminutive colorectal polyps using computer-aided analysis.Gastroenterology. 2018; 154: 568-575Abstract Full Text Full Text PDF PubMed Scopus (64) Google Scholar PolypsColonoscopy (WL or NBI)CNN7-fold cross-validation, dropout, early stopping8641 (4553/4088)cAfter data augmentation.1,330 (658/672)96.496.9/95.021Urban G. Tripathi P. Alkayali T. et al.Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy.Gastroenterology. 2018; 155: 1069-1078.e8Abstract Full Text Full Text PDF PubMed Scopus (71) Google Scholar PolypsColonoscopy (WL or NBI)CNN—788 (205/583): 602 training dataset, 186 test dataset78.092.3/62.522Renner J. Phlipsen H. Haller B. et al.Optical classification of neoplastic colorectal polyps—a computer-assisted approach (the COACH study).Scand J Gastroenterol. 2018; 53: 1100-1106Crossref PubMed Scopus (1) Google Scholar PolypsColonoscopyCNN—5545 (1911/3634)27,113 (21,572/5541)AUROC, 0.9894.4/95.923Wang P. Xiao X. Brown J.R.G. et al.Development and validation of a deep-learning algorithm for the detection of polyps during colonoscopy.Nat Biomed Eng. 2018; 2: 741Crossref PubMed Scopus (38) Google Scholar PolypsLinked color imaging colonoscopyGaussian mixture model—208 (69/139) from 112 patients181 (66/115) from 91 patients78.483.3/70.124Min M. Su S. He W. et al.Computer-aided diagnosis of colorectal polyps using linked color imaging colonoscopy to predict histology.Sci Rep. 2019; 9: 2881Crossref PubMed Scopus (0) Google Scholar PolypsEndocytoscopy (NBI and methylene blue)SVM—61,925466 (175/287/4 lost)96.593.8/91.025Mori Y. Kudo S.-E. Misawa M. et al.Real-time use of artificial intelligence in identification of diminutive polyps during colonoscopy: a prospective study.Ann Intern Med. 2018; 169: 357-366Crossref PubMed Scopus (50) Google Scholar PolypsEndocytoscopy (NBI)SVM—1661 (448 non-neoplasms/1213 neoplasms)173 (49 non-neoplasms/124 neoplasms)87.894.3/71.426Misawa M. Kudo S. Mori Y. et al.Accuracy of computer-aided diagnosis based on narrow-band imaging endocytoscopy for diagnosing colorectal lesions: comparison with experts.Int J Comp Assist Radiol Surg. 2017; 12: 757-766Crossref PubMed Scopus (0) Google Scholar PolypsWCE (colon)SVM—1000 (800/200)500 (400/100)95.091.0/95.227Romain O, Histace A, Silva J, et al. Towards a multimodal wireless video capsule for detection of colonic polyps as prevention of colorectal cancer. Proceedings of the 13th IEEE International Conference on BioInformatics and BioEngineering; November 10–13, 2013; Chania, Greece; pp 1–6.Google Scholar PolypsWCE (colon)MLPNonmaxima suppression31,600 (30,000/1600)cAfter data augmentation.30,540 (30,000/540)cAfter data augmentation.80.0NA28David E, Boia R, Malaescu A, et al. Automatic colon polyp detection in endoscopic capsule images. Proceedings of the International Symposium on Signals, Circuits and Systems ISSCS2013; July 11–12, 2013; Iasi, Romania; pp 1–4.Google Scholar PolypsWCE (colon)Binary—18,968 (18,738/230 corresponding to 16 polyps)NA81.2/90.229Mamonov A.V. Figueiredo I.N. Figueiredo P.N. et al.Automated polyp detection in colon capsule endoscopy.IEEE Trans Med Imaging. 2014; 33: 1488-1502Crossref PubMed Scopus (0) Google Scholar PolypsWCE (colon) or colonoscopyCNN—7910169596.497.1/93.330Blanes-Vidal V. Baatrup G. Nadimi E.S. Addressing priority challenges in the detection and assessment of colorectal polyps from capsule endoscopy and colonoscopy in colorectal cancer screening using machine learning.Acta Oncol. 2019; 58: S29-S36Crossref PubMed Scopus (1) Google ScholarFrom 124 patients without and 131 patients with polyps CRCColonoscopyCNN3-fold cross-validation9942 (5124 cTis+cT1a/4818 cT1b)cAfter data augmentation.5022 (2604 cTis+cT1a/2418 cT1b)81.267.5/89.031Ito N. Kawahira H. Nakashima H. et al.Endoscopic diagnostic support system for cT1b colorectal cancer using deep learning.Oncology. 2019; 96: 44-50Crossref PubMed Scopus (0) Google Scholar CRCConfocal laser endomicroscopy2-layer NNEarly stopping1035 (356/679)84.51.17 (cross-entropy)32Ştefănescu D. Streba C. Cârţână E.T. et al.Computer aided diagnosis for confocal laser endomicroscopy in advanced colorectal adenocarcinoma.PLoS One. 2016; 11e0154863Crossref PubMed Scopus (17) Google Scholar725155 CRCEndocytoscopySVM—5543 (2,506 non-neoplasms, 2,667 adenomas, 370 cancers)200 (100 adenomas, 100 cancers)94.189.4/98.933Takeda K. Kudo S.-E. Mori Y. et al.Accuracy of diagnosing invasive colorectal cancer using computer-aided endocytoscopy.Endoscopy. 2017; 49: 798-802Crossref PubMed Scopus (22) Google Scholar CRCEMRClassification and regression trees, LR or RF—263,879 (262,587/1,292)AUROC, 0.8964.2/90.03