Stratified analysis in comparative effectiveness studies that emulate randomized trials

Abstract Purpose For observational cohort studies that employ matching by propensity scores ( PS ), preliminary stratification by consequential predictors of outcome better emulates stratified randomization and potentially reduces variance and bias through relaxed dependence on modeling assumptions. We assessed the impact of pre‐stratification in two real‐life examples. For both, prior evidence from placebo‐controlled randomized clinical trials (RCTs) suggested small or no risk reduction, but observational analysis suggested protection, presumably the result of confounding bias. Study Design and Setting The study populations consisted of Medicare beneficiaries (2014‐18) with type 2 diabetes initiating either (i) empagliflozin versus dipeptidyl peptidase‐4 inhibitors (DPP‐4i) or (ii) empagliflozin versus glucagon‐like peptide‐1 receptor agonists (GLP‐1RA). The outcome was myocardial infarction or stroke. We estimated hazard ratios (HR) and rate differences (RD) after controlling for 143 pre‐exposure covariates via 1:1 PS matching after (1) PS estimation in the total cohort (total‐cohort PS‐matching) and (2) PS estimation separately by baseline cardiovascular disease (stratified PS matching). Results Stratified PS matching resulted in HRs that exceeded those from total‐cohort PS‐matching by 13% and 9%, respectively, for the comparisons of empagliflozin to DPP‐4i and GLP‐1RA. Against both comparators, HRs and RDs after stratified PS matching were closer to the null, with slightly higher variances (2%–3%) than those after total‐cohort PS matching. Conclusion Stratified PS matching produced effect estimates closer to the expected trial findings than total‐cohort PS matching. The price paid in increased variance was minimal.