Assessing the Quality of Metagenomic Next-Generation Sequencing for Pathogen Detection in Lower Respiratory Infections

Abstract Background Laboratory-developed metagenomic next-generation sequencing (mNGS) assays are increasingly being used for the diagnosis of infectious disease. To ensure comparable results and advance the quality control for the mNGS assay, we initiated a large-scale multicenter quality assessment to scrutinize the ability of mNGS to detect pathogens in lower respiratory infections. Methods A reference panel containing artificial microbial communities and real clinical samples was used to assess the performance of 122 laboratories. We comprehensively evaluated the reliability, the source of false-positive and false-negative microbes, as well as the ability to interpret the results. Results A wide variety of weighted F1-scores was observed across 122 participants, with a range from 0.20 to 0.97. The majority of false positive microbes (68.56%, 399/582) were introduced from “wet lab.” The loss of microbial sequence during wet labs was the chief cause (76.18%, 275/361) of false-negative errors. When the human context is 2 × 105 copies/mL, most DNA and RNA viruses at titers above 104 copies/mL could be detected by >80% of the participants, while >90% of the laboratories could detect bacteria and fungi at titers lower than 103 copies/mL. A total of 10.66% (13/122) to 38.52% (47/122) of the participants could detect the target pathogens but failed to reach a correct etiological diagnosis. Conclusions This study clarified the sources of false-positive and false-negative results and evaluated the performance of interpreting the results. This study was valuable for clinical mNGS laboratories to improve method development, avoid erroneous results being reported, and implement regulatory quality controls in the clinic.