PupilHeart: Heart Rate Variability Monitoring via Pupillary Fluctuations on Mobile Devices

Heart disease has now become a very common and impactful disease, which can actually be easily avoided if treatment is intervened at an early stage. Thus, daily monitoring of heart health has become increasingly important. Existing mobile heart monitoring systems are mainly based on seismocardiography (SCG) or photoplethysmography (PPG). However, these methods suffer from inconvenience and additional equipment requirements, preventing people from monitoring their hearts in any place at any time. Inspired by our observation of the correlation between pupil size and heart rate variability (HRV), we consider using the pupillary response when a user unlocks his/her phone using facial recognition to infer the user's HRV during this time, thus enabling heart monitoring. To this end, we propose a computer vision-based mobile HRV monitoring framework-PupilHeart, designed with a mobile terminal and a server side. On the mobile terminal, PupilHeart collects pupil size change information from users when unlocking their phones through the front-facing camera. Then, the raw pupil size data is preprocessed on the server side. Specifically, PupilHeart uses a 1-D convolutional neural network (1-D CNN) to identify time series features associated with HRV. In addition, PupilHeart trains a recurrent neural network (RNN) with three hidden layers to model pupil and HRV. Employing this model, PupilHeart infers users' HRV to obtain their heart condition each time they unlock their phones. We prototype PupilHeart and conduct both experiments and field studies to fully evaluate effectiveness of PupilHeart by recruiting 60 volunteers. The overall results show that PupilHeart can accurately predict the user's HRV.