Reporting of artificial intelligence prediction models

Data-driven technologies that form the basis of the digital health-care revolution provide potentially important opportunities to deliver improvements in individual care and to advance innovation in medical research. Digital health technologies include mobile devices and health apps (m-health), e-health technology, and intelligent monitoring. Behind the digital health revolution are also methodological advancements using artificial intelligence and machine learning techniques. Artificial intelligence, which encompasses machine learning, is the scientific discipline that uses computer algorithms to learn from data, to help identify patterns in data, and make predictions. A key feature underpinning the excitement behind artificial intelligence and machine learning is their potential to analyse large and complex data structures to create prediction models that personalise and improve diagnosis, prognosis, monitoring, and administration of treatments, with the aim of improving individual health outcomes. Prediction models to support clinical decision making have existed for decades, and these include well known tools such as the Framingham Risk Score, 1 Wilson PW D'Agostino RB Levy D Belanger AM Silbershatz H Kannel WB Prediction of coronary heart disease using risk factor categories. Circulation. 1998; 97: 1837-1847 Crossref PubMed Scopus (7437) Google Scholar QRISK3, 2 Hippisley-Cox J Coupland C Brindle P Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ. 2017; 357: j2099 Crossref PubMed Scopus (622) Google Scholar Model for End-stage Liver Disease, 3 Malinchoc M Kamath PS Gordon FD Peine CJ Rank J ter Borg PC A model to predict poor survival in patients undergoing transjugular intrahepatic portosystemic shunts. Hepatology. 2000; 31: 864-871 Crossref PubMed Scopus (2120) Google Scholar ABCD 2 Hippisley-Cox J Coupland C Brindle P Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ. 2017; 357: j2099 Crossref PubMed Scopus (622) Google Scholar score, 4 Johnston SC Rothwell PM Nguyen-Huynh MN et al. Validation and refinement of scores to predict very early stroke risk after transient ischaemic attack. Lancet. 2007; 369: 283-292 Summary Full Text Full Text PDF PubMed Scopus (1037) Google Scholar and the Nottingham Prognostic Index. 5 van Gorp MJ Steyerberg EW van der Graaf Y Decision guidelines for prophylactic replacement of Bjork-Shiley convexo-concave heart valves: impact on clinical practice. Circulation. 2004; 109: 2092-2096 Crossref PubMed Scopus (14) Google Scholar Health-care professionals, medical researchers, policy makers, guideline developers, patients, and members of the general public are all potential users of prediction models. The number of prediction model studies is increasing rapidly, with hundreds of different models being developed for some of the same targeted populations and outcomes. 6 Damen JAAG Hooft L Schuit E et al. Prediction models for cardiovascular disease risk in the general population: systematic review. BMJ. 2016; 353: i2416 Crossref PubMed Scopus (427) Google Scholar , 7 Tangri N Kitsios GD Inker LA et al. Risk prediction models for patients with chronic kidney disease: a systematic review. Ann Intern Med. 2013; 158: 596-603 Crossref PubMed Scopus (134) Google Scholar Extension of the CONSORT and SPIRIT statementsWe read with great interest the proposal made by Gary Collins and Karel Moons1 to develop a version of the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) statement that is specific to machine learning (ML), to be known as TRIPOD-ML.2 We agree that the understandable excitement around ML-enabled technologies should not overrule the need for robust scientific evaluation, and ML prediction model studies should adopt established guidance for reporting. Full-Text PDF Walking the tightrope of artificial intelligence guidelines in clinical practiceOver the past few months, there has been a wave of digital health guidelines and whitepapers issued by regulators, institutes, and organisations worldwide. In the field of artificial intelligence (AI), EU guidelines, published in April, promote the development of trustworthy AI across all disciplines, while a US Food and Drug Administration (FDA) whitepaper proposes a regulatory framework for constantly developing software in health care. Guidelines from the National Institution of Health and Care Excellence (NICE) tackle the level of evidence required for a new digital health intervention, and NHSX and Public Health England have both reported their intention to produce their own AI guidelines. Full-Text PDF Open Access