Skin Reflection Angle useful for Region of Interest Se-lection in Camera-based Heart Rate Estimation?

Maurice Rohr1, Philipp Witulla2, Christoph Hoog Antink3
1Technische Universität Darmstadt, 2Technical University of Darmstadt, 3TU Darmstadt


Abstract

Background: Photoplethysmography Imaging (PPGI) is considered a prospective non-contact replacement for existing methods of heart rate (HR), oxygenation, and even blood pressure estimation. The robust selection of pixels with a strong pulse signal is the most important processing step for realistic scenarios that include significant movement and illumination changes. Approaches for pixel selection include simple face detection, skin segmentation, and experimentally/physiologically motivated landmark-based region selection. These methods do not consider scene geometry or illumination. Hence, we evaluate if the angle between the camera sensor and the skin might be a relevant variable as suggested in the literature. Methods: We employ BlazeFace for face and facial landmark detection. Using these landmarks, we segment forehead and cheek regions, which are considered optimal for pulse extraction. Furthermore, we use 3D-coordinate estimates of the landmarks to calculate the angle between the camera axis and the skin normal. To quantify improvements from only using skin parallel to the image plane, we use five different thresholds on the angle between 15° and 90° to reject additional sub-regions. We evaluate this skin region selection scheme on commonly used PPGI signal extraction algorithms and six openly available video datasets with pulse reference, totaling 290 subjects. Results: For the CHROM algorithm thresholds at 45°/60° marginally improve the mean absolute error of heart rate estimation by 0.1 BPM compared to using the cheek/forehead region. Most other algorithms do not benefit from angle thresholds. Conclusion: The skin-to-camera angle can be used to select regions for pulse estimation. However, the method is computationally expensive and, based on close inspection of the data where it performed well and the fact that small angles below 15° generally do not lead to improvements, it seems unlikely that the skin-to-camera angle is a relevant confounder to quality in PPGI estimation.