An interactive greeting system using convolutional neural networks for emotion recognition

The convolutional neural network (CNN) has been progressively applied in face recognition and various applications. This paper presents an interactive greeting system that combines face identification, emotion recognition, and funny 3D animation. The proposed system recognizes a user by a face-CNN and interacts with the user according to the recognized mood. The animated 3D girl plays a motion-dependent video according to the recognized user’s emotion. First, we employ a Viola-Jones algorithm to segment the areas around the face, eyes, and mouth in a captured image. The face-CNN recognizes a known person. An eye-CNN and a mouth-CNN recognize the emotion. Finally, a 3D animation is played according to the identified emotion. Experimental results show that the proposed interactive greeting system recognizes target persons well, identifies the emotion adequately, and provides users to enjoy funny interaction.