Artificial intelligence to evaluate postoperative pain based on facial expression recognition

Abstract Background Pain intensity evaluation by self‐report is difficult and biased in non‐communicating people, which may contribute to inappropriate pain management. The use of artificial intelligence (AI) to evaluate pain intensity based on automated facial expression analysis has not been evaluated in clinical conditions. Methods We trained and externally validated a deep‐learning system (ResNet‐18 convolutional neural network) to identify and classify 2810 facial expressions of 1189 patients, captured before and after surgery, according to their self‐reported pain intensity using numeric rating scale (NRS, 0–10). AI performances were evaluated by accuracy (concordance between AI prediction and patient‐reported pain intensity), sensitivity and specificity to diagnose pain ≥4/10 and ≥7/10. We then confronted AI performances with those of 33 nurses to evaluate pain intensity from facial expression in the same situation. Results In the external testing set (120 face images), the deep learning system was able to predict exactly the pain intensity among the 11 possible scores (0–10) in 53% of the cases with a mean error of 2.4 points. Its sensitivities to detect pain ≥4/10 and ≥7/10 were 89.7% and 77.5%, respectively. Nurses estimated the right NRS pain intensity with a mean accuracy of 14.9% and identified pain ≥4/10 and ≥7/10 with sensitivities of 44.9% and 17.0%. Conclusions Subject to further improvement of AI performances through further training, these results suggest that AI using facial expression analysis could be used to assist physicians to evaluate pain and detect severe pain, especially in people not able to report appropriately their pain by themselves. Significance These original findings represent a major step in the development of a fully automated, rapid, standardized and objective method based on facial expression analysis to measure pain and detect severe pain.