Evaluating BLOOMY and SOFA scores in hospitalised patients

We congratulate Evelina Tacconelli and colleagues1Tacconelli E Göpel S Gladstone BP et al.Development and validation of BLOOMY prediction scores for 14-day and 6-month mortality in hospitalised adults with bloodstream infections: a multicentre, prospective, cohort study.Lancet Infect Dis. 2022; (published online Jan 19.)https://doi.org/10.1016/S1473-3099(21)00587-9Summary Full Text Full Text PDF PubMed Scopus (3) Google Scholar on developing the Bloodstream Infection due to Multidrug-resistant Organisms: Multicenter Study on Risk Factors and Clinical Outcomes (BLOOMY) prediction scores. Among patients admitted to hospital with bloodstream infection, the BLOOMY 14-day score had a C statistic of 0·873 for mortality, while the simplified quick BLOOMY score had a C statistic of 0·828. Strengths of this commendable study include prospective multicentre data collection. However, the Article raises important questions. First, because of variable patient-level baseline risks, subgroup analyses are essential to determine the degree of heterogeneity in these variables’ predictive performance across different populations. Although the Methods describe subgroup analyses, we could not find these results in the main Article or its appendix. Second, the BLOOMY and quick BLOOMY scores were compared only indirectly to the Sequential Organ Failure Assessment (SOFA) and quick SOFA (qSOFA) scores1Tacconelli E Göpel S Gladstone BP et al.Development and validation of BLOOMY prediction scores for 14-day and 6-month mortality in hospitalised adults with bloodstream infections: a multicentre, prospective, cohort study.Lancet Infect Dis. 2022; (published online Jan 19.)https://doi.org/10.1016/S1473-3099(21)00587-9Summary Full Text Full Text PDF PubMed Scopus (3) Google Scholar due to unavailability of respiratory rate. Such indirect comparisons are often not valid due to differential case mix and differences in clinical practices between model development populations.2Collins GS Moons KGM Comparing risk prediction models.BMJ. 2012; 344e3186Crossref Scopus (77) Google Scholar We are particularly interested in BLOOMY's performance among patients with cancer, because of the prevalence of bloodstream infection in this group3Hensley MK Donnelly JP Carlton EF Prescott HC Epidemiology and outcomes of cancer-related versus non-cancer-related sepsis hospitalizations.Crit Care Med. 2019; 47: 1310-1316Crossref PubMed Scopus (19) Google Scholar and the potential for short-term risk estimates to influence decisions on cancer-directed therapies and supportive care. We applied BLOOMY, quick BLOOMY, SOFA, and qSOFA to electronic health record data from a single-centre cohort of oncology patients meeting BLOOMY inclusion criteria from June 1, 2018, to June 30, 2021.4Lyons PG Klaus J McEvoy CA Westervelt P Gage BF Kollef MH Factors associated with clinical deterioration among patients hospitalized on the wards at a tertiary care hospital.J Oncol Pract. 2019; 15: e652-e665Crossref PubMed Scopus (10) Google Scholar We compared 14-day mortality C statistics (BLOOMY 14-day vs SOFA; quick BLOOMY vs qSOFA). Of 844 patients, 33 (4%) died within 14 days of blood culture collection. C statistics for 14-day mortality did not differ between BLOOMY (0·734 [95% CI 0·659–0·810]) and SOFA (0·721 [0·637–0·804]; p=0·75) or between quick BLOOMY (0·739 [0·664–0·813]) and qSOFA scores (0·712 [0·629–0·794]; p=0·30). Our findings have important implications. First, although cancer-related bloodstream infection has been identified as a risk factor for mortality,3Hensley MK Donnelly JP Carlton EF Prescott HC Epidemiology and outcomes of cancer-related versus non-cancer-related sepsis hospitalizations.Crit Care Med. 2019; 47: 1310-1316Crossref PubMed Scopus (19) Google Scholar mortality was lower in our cohort than Tacconelli and colleagues’ study. Second, we found lower discrimination for BLOOMY than for SOFA and quick BLOOMY than for qSOFA in our cohort. These findings probably indicate a so-called dataset shift—ie, differential case mix, epidemiology, and practices between cohorts.5Finlayson SG Subbaswamy A Singh K et al.The clinician and dataset shift in artificial intelligence.N Engl J Med. 2021; 385: 283-286Crossref PubMed Scopus (36) Google Scholar We hope Tacconelli and colleagues can report their malignancy-specific results to contextualise our findings. Finally, BLOOMY and quick BLOOMY did not outperform SOFA and qSOFA in our cohort. The newly developed scores, despite using many of the same predictors, are more complex than SOFA and qSOFA. Without improvement within the context of bloodstream infection, the value of using such models is unclear. Thus, we urge further external validation of BLOOMY and quick BLOOMY, particularly among patients with cancer, before widespread adoption. We declare no competing interest. The code for this project is freely available upon request to the corresponding author. Development and validation of BLOOMY prediction scores for 14-day and 6-month mortality in hospitalised adults with bloodstream infections: a multicentre, prospective, cohort studyThe BLOOMY scores showed good discrimination and predictive values and could support the development of protocols to manage bloodstream infections and also help to estimate the short-term and long-term burdens of bloodstream infections. Full-Text PDF Evaluating BLOOMY and SOFA scores in hospitalised patients – Authors' replyWe thank Nicole Benzoni and colleagues for their Correspondence and for sharing the results of an assessment of the 14-day mortality Bloodstream Infection due to Multidrug-resistant Organisms: Multicenter Study on Risk Factors and Clinical Outcomes (BLOOMY) score1 in a retrospective cohort of US-based hospitalised patients with cancer and bloodstream infections. We are pleased to see that in the C statistics the 14-day BLOOMY score in the assessed population was slightly better than the Sequential Organ Failure Assessment (SOFA) score. Full-Text PDF