Abstract Background The associations of physical pre-fraily and frailty with bone fractures and the modified effect of sedentary lifestyle remain uncertain. This study was performed to explore the association of physical pre-frailty and frailty with risk of incident bone fractures; and test the modification effects of sedentary lifestyle and other risk factors. Methods This cohort study included 413,630 participants without bone fractures at baseline in the UK Biobank study between 2006 and 2010 and followed up to 2021. The mean age of the participants was 56.5 years. A total of 224,351 (54.2%) enrolled participants were female and 376,053 (90.9%) included participants were white. Three Cox regression models were constructed to analyze the association of pre-frailty and frailty with total fractures, hip fractures, vertebrae fractures and other fractures. Results As compared with the physical non-frailty group, the multivariate adjusted Hazard Ratios (HRs) were 1.17 (95% CI: 1.14 to 1.21) and 1.63 (95% CI: 1.53 to 1.74) for the physical pre-frailty group and frailty group, respectively (P-trend<0.001). In addition, we found that sedentary behavior time significantly accentuated the associations of physical pre-frailty and frailty with total fractures (P-interaction<0.001), hip fractures (P-interaction=0.013) and other fractures (P-interaction<0.001). Conclusions Our results indicate that physical pre-frailty and frailty are related to higher risks of bone fractures; such association was more pronounced among those with longer sedentary behavior time.