Machine Learning and Electronic Health Records: A Paradigm Shift

Back to table of contents Previous article Next article EditorialsFull AccessMachine Learning and Electronic Health Records: A Paradigm ShiftDaniel E. Adkins, Ph.D.Daniel E. AdkinsSearch for more papers by this author, Ph.D.Published Online:1 Feb 2017https://doi.org/10.1176/appi.ajp.2016.16101169AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinked InEmail In this issue of the Journal, Barack-Corren et al. (1) use machine learning methods to build a highly predictive model of suicidal behavior using longitudinal electronic health records (EHRs). They do so using a well-established probability-based machine learning algorithm, the naive Bayesian classifier, to mine through approximately 1.7 million patient records, spanning 15 years (1998–2012), from two large Boston hospitals. After training the naive Bayesian classifier model on a randomly selected half of the data, the predictive ability of the model was assessed on the second half, yielding accurate (35%−49% sensitivity at 90%−95% specificity) and, critically, early (3–4 years in advance on average) prediction of patients’ future suicidal behavior. In this, the authors benefitted from access to a large and high-quality EHR database and chose an appropriate, and powerful, analytical method in the naive Bayesian classifier. Furthermore, the research has clear clinical applications in the potential for early detection warnings via physician EHR notices. Beyond such specifics, the study has broader significance in its demonstration of how the atheoretical machine learning approaches popular in Silicon Valley can successfully mine clinical insights from an exponentially growing body of EHR data. It also hints toward a future in which machine learning of big medical data may become a ubiquitous component of clinical research and practice—a prospect that some are uncomfortable with.While the pace at which machine learning applications diffuse into clinical research and practice remains to be seen, methodological development in the machine learning field continues to accelerate. And this suggests one primary limitation of the current study. That is, while the naive Bayesian classifier is well-suited to the current application, it is an older and remarkably simple method by machine learning standards. Fundamentally, the naive Bayesian classifier is a direct application of Bayes’ theorem, simply calculating the product of the prior probability of the outcome of interest (e.g., suicidal behavior) and the probabilities for each predictor in the data conditional on the outcome of interest (2). This analytical simplicity contrasts sharply with more advanced machine learning techniques, including neural nets, deep learning, and ensemble methods, which achieve notable increases in prediction compared with naive Bayesian classifier, but are black boxes in terms of estimation, as their models are extremely large, complex, and characterized by “hidden layers” (3, 4). So, while there is ample room for improved prediction accuracy in Barack-Corren et al.’s approach, such gains would likely come at the expense of interpretability and inference. Thus, their selection of the naive Bayesian classifier has the further, unintended merit of providing an unusually lucid, accessible introduction to machine learning for many researchers and clinicians.Another limitation, perhaps strategic on Barack-Corren et al.’s part, is the use of a limited set of standard ICD-9 codes and search terms as predictors, versus performing natural language processing of the full semistructured data of the EHR. This analytical decision is a significant limitation, as it drastically reduces the analysis feature space (i.e., the number of predictors considered), which generally results in poorer prediction given data of this size (5, 6). While the authors do not give a precise number of predictors used in their analysis, we can safely assume it is at least an order of magnitude less than what would be possible using natural language processing techniques. However, this again raises the issue of model interpretability, as natural language processing approaches may identify highly predictive features that offer no clear interpretation or clinical significance (6). Contrast that opacity with Barack-Corren et al.’s list of the top 100 predictors in their naive Bayesian classifier (see Table S2 in the article’s online data supplement), which summarizes a wealth of clinical insight, and we again see the precision advantages of more sophisticated approaches counterbalanced by the interpretability of simpler models like Barack-Corren et al.’s naive Bayesian classifier. This tradeoff is not specific to the current topic, instead it is a pervasive aspect of machine learning—a continuum of inference versus prediction that is traversed when moving from simpler approaches, like Barack-Corren et al.’s naive Bayesian classifier, to more advanced, opaque approaches, including neural nets and deep learning (7, 8).Stepping back from the technical aspects of machine learning, this study provides an opportunity to reflect on the trend of the field toward increasingly data-driven approaches. Regardless of the promise of machine learning of EHR, it would be unwise to endorse the approach without first considering the various professional, ethical, and legal issues accompanying the potential improvements in diagnosis and treatment. From the perspective of praxis, it is noteworthy that the approach, carried to its logical conclusion, is fundamentally atheoretical, which marks a stark departure from conventional clinical paradigms built primarily on evidence-based causal models (9). Furthermore, for some it may seem like a slippery slope toward ceding power in the clinic to algorithms and devaluing clinician experience and judgment. But I would note that the majority of a clinician’s function would not, and indeed could not, be encroached upon by data-driven analytics. Rather, increasing the role of machine learning applications to EHRs would provide additional inputs for the clinician to consider in making diagnostic and treatment decisions. In this way, the emergence of machine learning EHR prediction may be seen as analogous to the development of imaging, genetic, or any other new source of highly informative medical data. Additionally, there are ethical and legal issues surrounding the mining of EHR, including protecting the patient population from adverse consequences stemming from the analysis of their data. This suggests potentially problematic dynamics if, for instance, EHR data and analytics are accessed by insurance companies, who may use the data to discriminate against patients in the marketplace. This risk is compounded by the possibility of black box machine learning methods inadvertently identifying stratifying criteria that we as a society find unacceptable.While ethical arguments for the use of participant data often take the form of efforts to limit access to data, as in the well-justified attention paid to patient privacy and nondisclosure, a powerful argument for the opposite exists in regard to enhancing public benefit through the analysis of EHR data. That is, as the data are often collected using some combination of patient permission and government funding, it may be reasonable to consider public benefit as a goal, or even an obligation, in the collection and analysis of the data. Although this does not argue against private sector activity, it does support a concerted effort to consolidate data and analyses funded by federal research dollars into a public resource—and what a tremendous resource a centralized archive of EHR data staffed with a cadre of machine learning analysts could be. Currently, this possibility is prevented by data fragmentation, as most EHR data are presently proprietary (10, 11), but this could change with leadership from federal entities. And we have good precedent from the National Institutes of Health and Veterans Affairs regarding safeguarding, and maximizing benefit from, comparable archives (e.g., dbGaP [database of Genomics and Phenotypes]).In summary, as demonstrated by Barack-Corren et al., the application of machine learning methods to EHRs, and the potential of extending such analyses to other sources of big medical data (e.g., genomics and imaging), could generate enormous—yes, even paradigm-shifting—returns in improved diagnosis and treatment. What remains unclear is the pace at which these benefits will be realized, as well as who the primary beneficiaries will be.From the Departments of Psychiatry and Sociology, University of Utah, Salt Lake City, Utah.Address correspondence to Dr. Adkins ([email protected]).Supported by the National Institute of Mental Health (grant K01MH093731) and the University of Utah, Consortium for Families and Health Research.The author reports no financial relationships with commercial interests.The author thanks Jason D. Thomas and Anna R. Docherty for assistance and critique.References1 Barak-Corren Y, Castro VM, Javitt S, et al.: Predicting Suicidal Behavior From Longitudinal Electronic Health Records. Am J Psychiatry 2017; 174:154–162Link, Google Scholar2. Friedman N, Geiger D, Goldszmidt M: Bayesian network classifiers. Mach Learn 1997; 29:154–162Crossref, Google Scholar3. LeCun Y, Bengio Y, Hinton G: Deep learning. Nature 2015; 521:436–444Crossref, Medline, Google Scholar4. Schmidhuber J: Deep learning in neural networks: an overview. Neural Netw 2015; 61:85–117Crossref, Medline, Google Scholar5. Jurafsky D, Martin JH: Speech and Language Processing, 2nd ed. Upper Saddle River, NJ, Prentice Hall, 2008Google Scholar6. Lin J, Dyer C: Data-Intensive Text Processing with MapReduce. San Rafael, Calif, Morgan and Claypool Publishers, 2010Crossref, Google Scholar7. Breiman L: Statistical modeling: the two cultures. Stat Sci 2001; 16:199–231Crossref, Google Scholar8. Kelleher JD, Namee BM, D’Arcy A: Fundamentals of Machine Learning for Predictive Data Analytics. Cambridge, Mass, The MIT Press, 2015Google Scholar9. Greenland S, Pearl J, Robins JM: Causal diagrams for epidemiologic research. Epidemiology 1999; 10:37–48Crossref, Medline, Google Scholar10. Hall MA: Property, privacy and the pursuit of integrated electronic medical records. Iowa Law Rev 2010; 2010:631–663Google Scholar11. Jensen PB, Jensen LJ, Brunak S: Mining electronic health records: towards better research applications and clinical care. Nat Rev Genet 2012; 13:395–405Crossref, Medline, Google Scholar FiguresReferencesCited byDetailsCited byA Framework for Automatic Clustering of EHR Messages Using a Spatial Clustering Approach30 January 2023 | Healthcare, Vol. 11, No. 3Does AI explainability affect physicians’ intention to use AI?International Journal of Medical Informatics, Vol. 168Machine Learning Models for the Prediction of Renal Failure in Chronic Kidney Disease: A Retrospective Cohort Study11 October 2022 | Diagnostics, Vol. 12, No. 10Ten simple rules for organizing a special session at a scientific conference25 August 2022 | PLOS Computational Biology, Vol. 18, No. 8Predicting the Need for Therapeutic Intervention and Mortality in Acute Pancreatitis: A Two-Center International Study Using Machine Learning11 April 2022 | Journal of Personalized Medicine, Vol. 12, No. 4A Survey on Publicly Available Open Datasets Derived From Electronic Health Records (EHRs) of Patients with Neuroblastoma4 October 2022 | Data Science Journal, Vol. 21, No. 1An Efficient Ensemble Model for Various Scale Medical DataComputers, Materials & Continua, Vol. 73, No. 1Leveraging the potential of machine learning for assessing vascular ageing: state-of-the-art and future research18 October 2021 | European Heart Journal - Digital Health, Vol. 2, No. 4What is Interpretability?12 November 2020 | Philosophy & Technology, Vol. 34, No. 4Towards Validating the Effectiveness of Obstructive Sleep Apnea Classification from Electronic Health Records Using Machine Learning27 October 2021 | Healthcare, Vol. 9, No. 11Utilizing support vector machines for predictive analytics in chronic kidney diseasesMaterials Today: Proceedings, Vol. 13Machine Learning Prediction Models for Chronic Kidney Disease Using National Health Insurance Claim Data in Taiwan7 May 2021 | Healthcare, Vol. 9, No. 5Big Brain Data: On the Responsible Use of Brain Data from Clinical and Consumer-Directed Neurotechnological Devices19 May 2018 | Neuroethics, Vol. 14, No. 1Natural language processing and machine learning of electronic health records for prediction of first-time suicide attempts17 March 2021 | JAMIA Open, Vol. 4, No. 1An Early Disease Prediction and Risk Analysis of Diabetic Mellitus using Electronic Medical RecordsIOP Conference Series: Materials Science and Engineering, Vol. 1085, No. 1History, current status, and future directions of artificial intelligenceA Comprehensive Review on Medical Diagnosis Using Machine LearningComputers, Materials & Continua, Vol. 67, No. 2Robust data‐driven identification of risk factors and their interactions: A simulation and a study of parental and demographic risk factors for schizophrenia10 June 2020 | International Journal of Methods in Psychiatric Research, Vol. 29, No. 4Genome-Wide Association Study of Suicide Death and Polygenic Prediction of Clinical AntecedentsAnna R. Docherty, Ph.D., Andrey A. Shabalin, Ph.D., Emily DiBlasi, Ph.D., Eric Monson, M.D., Niamh Mullins, Ph.D., Daniel E. Adkins, Ph.D., Silviu-Alin Bacanu, Ph.D., Amanda V. Bakian, Ph.D., Sheila Crowell, Ph.D., Danli Chen, Ph.D., Todd M. Darlington, Ph.D., William B. Callor, M.S., Erik D. Christensen, M.D., Douglas Gray, M.D., Brooks Keeshin, M.D., Michael Klein, M.S., John S. Anderson, B.S., Leslie Jerominski, M.S., Caroline Hayward, Ph.D., David J. Porteous, Ph.D., Andrew McIntosh, M.D., Qingqin Li, Ph.D., Hilary Coon, Ph.D.1 October 2020 | American Journal of Psychiatry, Vol. 177, No. 10Beyond the Randomized Clinical Trial: Innovative Data Science to Close the Pediatric Evidence GapClinical Pharmacology & Therapeutics, Vol. 107, No. 4The emerging landscape of health research based on biobanks linked to electronic health records: Existing resources, statistical challenges, and potential opportunities20 December 2019 | Statistics in Medicine, Vol. 39, No. 6Navigating the challenges of performing anesthesia handoffs and conducting anesthesia handoff research19 November 2019 | International Anesthesiology Clinics, Vol. 58, No. 1Identifying the Medical Lethality of Suicide Attempts Using Network Analysis and Deep Learning: Nationwide Study9 July 2020 | JMIR Medical Informatics, Vol. 8, No. 7Interpreting patient-Specific risk prediction using contextual decomposition of BiLSTMs: application to children with asthma8 November 2019 | BMC Medical Informatics and Decision Making, Vol. 19, No. 1Developing a FHIR-based EHR phenotyping framework: A case study for identification of patients with obesity and multiple comorbidities from discharge summariesJournal of Biomedical Informatics, Vol. 99Predicting Suicide Attempts and Suicide Deaths Following Outpatient Visits Using Electronic Health RecordsGregory E. Simon, M.D., M.P.H., Eric Johnson, M.S., Jean M. Lawrence, Sc.D., Rebecca C. Rossom, M.D., M.S., Brian Ahmedani, Ph.D., Frances L. Lynch, Ph.D., Arne Beck, Ph.D., Beth Waitzfelder, Ph.D., Rebecca Ziebell, Robert B. Penfold, Ph.D., Susan M. Shortreed, Ph.D.24 May 2018 | American Journal of Psychiatry, Vol. 175, No. 10 Volume 174Issue 2 February 01, 2017Pages 93-94 Metrics KeywordsDiagnosis And ClassificationEpidemiologyEthicsStatisticsSuicideThe author thanks Jason D. Thomas and Anna R. Docherty for assistance and critique.PDF download History Accepted 1 October 2016 Published online 1 February 2017 Published in print 1 February 2017