Google researchers have developed a new deep-learning system that can passively measure a user’s heart rate through a smartphone’s front-facing camera during normal device use. The technology, described in recently published research, aims to bring medical-grade heart rate monitoring to billions of people without requiring any wearable hardware.
Resting heart rate is a key biomarker linked to cardiovascular health and long-term disease risk. High resting heart rate is associated with adverse cardiovascular events and certain chronic conditions. With roughly five billion smartphones worldwide already equipped with the necessary camera hardware, the potential for widespread passive health monitoring is significant.
How the System Works
The system, called passive heart rate monitoring (PHRM), uses the front-facing camera to record short video clips of the user’s face. A temporal shift convolutional neural network then analyzes these clips to estimate heart rate. The method relies on detecting subtle changes in light reflection caused by blood pulsing through the skin—a technique known as remote photoplethysmography (rPPG).
According to Google, the system achieves a mean absolute percentage error of less than 10%, meeting industry accuracy standards across all skin tones. “To our knowledge, PHRM marks the first large-scale demonstration of passive HR and daily RHR monitoring during everyday smartphone use,” said Eric S. Teasley, Product Manager, and Ming-Zer Poh, Staff Research Scientist at Google Research.
The researchers added: “As the only rPPG method to meet heart rate accuracy standards for people of all skin tones – even in unpredictable real-world conditions – it sets a new standard for the field. It also represents the first use of rPPG to estimate daily RHR, achieving wearable-level accuracy across all skin tones.”
Diverse Training Data and Real-World Testing
Previous studies in this area have often underrepresented people with dark skin, as melanin can make optical signals harder to detect. To address this, Google’s team built their model using more than 350,000 video clips from nearly 700 participants. The Monk Skin Tone Scale was used to ensure diverse representation: participants with light and medium skin tones each comprised at least 25% of the datasets, while those with dark skin tones made up at least 33%. This makes it the largest and most diverse rPPG study to date.
The system was tested in both laboratory and real-world conditions. In the lab, researchers recorded facial video and simultaneous electrocardiogram (ECG) data from 365 participants, and the PHRM system outperformed 15 leading published rPPG models. In a real-world study, 231 participants installed a data collection app on their phones and used them normally while wearing an ECG chest strap and a Fitbit tracker. The app captured an average of 231 video clips per day.
The researchers note that further optimization of camera exposure and handling of excessive head movement could improve performance. Google plans to make its data and modeling resources available to qualified researchers.