Educational Equity Through Combined Human-AI Personalization: A Propensity Matching Evaluation

Recent developments in combined human-computer tutoring systems show promise in narrowing math achievement gaps among marginalized students. We present an evaluation of the use of the Personalized Learning2, a hybrid tutoring approach whereby human mentoring and AI tutoring are combined to personalize learning with respect to students' motivational and cognitive needs. The approach assumes achievement gaps emerge from differences in learning opportunities and seeks to increase such opportunities for marginalized students through after-school programs, such as the Ready to Learn program. This program engaged diverse middle school students from three schools in an urban district. We compared achievement growth of 70 treatment students in this program with a control group of 380 students from the same district selected by propensity matching to have similar demographics and prior achievement. Based on standardized math assessments (NWEA Measures of Academic Progress) given one year apart, we found the gain of treatment students (6.8 points) was nearly double the gain of the control group (3.6 points). Further supporting the inference that greater learning was caused by the math-focused treatment and not by some selection bias, we found no significant differences in reading achievement between treatment and control participants. These results show promise that greater educational equity can be achieved at reasonable costs through after-school programs that combine the use of low-cost paraprofessional mentors and computer-based tutoring.