Video-based real-time monitoring for heart rate and respiration rate

During the last decade, the world faced many pandemics, causing medical service providers to struggle with diagnosing, following up with patients, keeping daily records, and eliminating infection spread. All of these factors force us to pay close attention in order to make vital sign measurements safer and easier. Respiration Rate (RR) and Heart Rate (HR) are the most measured signs for patients. Remote photoplethysmography (rPPG) is a video-based technique for HR monitoring so that telehealth can be easier. This paper proposes a new methodology for RR and HR estimation depending on non-contact techniques. The proposed architecture relies on monitoring the patients using a camera to view a video stream from which we can extract the rPPG waveform from individuals’ faces. The motion and color in the video are first magnified using Eulerian Video Magnification (EVM) and then analyzed in two stages, one for HR estimation and the other for RR estimation. For HR estimation, MediaPipe Face Mesh is employed to annotate the boundaries of the most suitable Region of Interest (ROI) from the face image in both RGB and HSV color modes. Then, the integral image for R and V channels, respectively, are computed. The proposed method is based on measuring fluctuations in the value resulting from the integral image, and can therefore extract HR. Whilst for RR estimation, MediaPipe Pose solution is used to annotate the position of specific landmarks on the chest, and then tracking the changes of these landmarks' positions with time. The performance of the proposed method is evaluated using COHFACE dataset. In HR experiments, the Mean Absolute Error (MAE) is 2.05 and 2.03 BPM (Beats per Minute), and the Pearson Correlation Coefficient (PCC) is 0.91 and 0.86 for RGB and HSV frames, respectively. In RR experiments, the MAE was 1.62 BrPM (Breaths per Minute) and the PCC is 0.45.