Evaluating Performance and Agreement of Coronary Heart Disease Polygenic Risk Scores

Importance Polygenic risk scores (PRSs) for coronary heart disease (CHD) are a growing clinical and commercial reality. Whether existing scores provide similar individual-level assessments of disease susceptibility remains incompletely characterized. Objective To characterize the individual-level agreement of CHD PRSs that perform similarly at the population level. Design, Setting, and Participants Cross-sectional study of participants from diverse backgrounds enrolled in the All of Us Research Program (AOU), Penn Medicine BioBank (PMBB), and University of California, Los Angeles (UCLA) ATLAS Precision Health Biobank with electronic health record and genotyping data. Exposures Polygenic risk for CHD from published PRSs and new PRSs developed separately from testing samples. Main Outcomes and Measures PRSs that performed population-level prediction similarly were identified by comparing calibration and discrimination of models of prevalent CHD. Individual-level agreement was tested with intraclass correlation coefficient (ICC) and Light κ. Results A total of 48 PRSs were calculated for 171 095 AOU participants. The mean (SD) age was 56.4 (16.8) years. A total of 104 947 participants (61.3%) were female. A total of 35 590 participants (20.8%) were most genetically similar to an African reference population, 29 801 (17.4%) to an admixed American reference population, 100 493 (58.7%) to a European reference population, and the remaining to Central/South Asian, East Asian, and Middle Eastern reference populations. There were 17 589 participants (10.3%) with and 153 506 participants without (89.7%) CHD. When included in a model of prevalent CHD, 46 scores had practically equivalent Brier scores and area under the receiver operator curves (region of practical equivalence ±0.02). Twenty percent of participants had at least 1 score in both the top and bottom 5% of risk. Continuous agreement of individual predictions was poor (ICC, 0.373 [95% CI, 0.372-0.375]). Light κ, used to evaluate consistency of risk assignment, did not exceed 0.56. Analysis among 41 193 PMBB and 53 092 ATLAS participants yielded different sets of equivalent scores, which also lacked individual-level agreement. Conclusions and Relevance CHD PRSs that performed similarly at the population level demonstrated highly variable individual-level estimates of risk. Recognizing that CHD PRSs may generate incongruent individual-level risk estimates, effective clinical implementation will require refined statistical methods to quantify uncertainty and new strategies to communicate this uncertainty to patients and clinicians.