Exercise-tracking apps are digital tools for delivering personalized behavioral interventions. Despite the growing usage of exercise applications, the efficacy of in-exercise app features in driving usage and athletic outcomes remains poorly understood. To remain competitive, sports organizations now need to leverage tracking tools to efficiently allocate resources and streamline training regimens and interventions for their core assets (i.e., athletes). In response to these operational needs, we examine two specific forms of such in-exercise interventions, namely performance feedback and social feedback. We conducted an 18-month-long field study with 1,037 uniformed group servicemen to assess the effect of these feedback types on running and usage outcomes. Results from the field study provided evidence that these two app features improved the servicemen’s running times and frequency of application usage, on average. Contrary to the common belief that more features are better, the joint usage of two feedback features does not produce additive effects. Tests at more granular levels suggest that users who received both feedback types in exercise episodes exhibit overconfidence behavior by participating in fewer subsequent exercises. The receipt of both feedback may be redundant and can cause user annoyance. Heterogeneity tests revealed that while performance feedback benefited most runners, social features were effective only for already stronger runners. Also, only positive social feedback had a significant impact on running performance. The results further indicate that performance feedback generated a slow but sustained increase in usage frequency, while social feedback spurred quick initial growth in usage but dwindled in effectiveness over time. Implications for theory and practice, as well as directions for further research, are discussed.