Remote Photoplethysmography (rPPG) API
What is rPPG and what can a camera actually measure?
Remote photoplethysmography (rPPG) estimates a pulse signal from tiny colour changes in skin recorded by an ordinary camera, with no contact and no wearable. Under good lighting and limited motion, published research supports pulse-rate estimation; breathing rate and higher-level indicators such as alertness are experimental and probabilistic. Hannes Bend is a named inventor on US patents 10,423,893 (2019) and 11,561,806 (2023) for camera-based sensing and adaptive response. Health, telehealth and fatigue-safety uses are possible applications only, requiring separate validation and regulatory review.
Remote photoplethysmography (rPPG) enables contactless physiological measurement using standard RGB cameras. By analysing subtle variations in skin reflectance caused by blood volume pulse changes, rPPG systems can estimate pulse rate under favourable conditions. Derived measures such as pulse-rate variability and breathing rate are more fragile, and anything beyond them — alertness, fatigue, stress — is an inference about a person, not a measurement of one.
Hannes Bend began exploring webcam-based rPPG systems between 2017–2018, integrating physiological signal detection into immersive environments and adaptive audiovisual systems.
In 2020, Hannes Bend led an international volunteer initiative involving more than 170 healthcare practitioners, researchers, data scientists, designers, developers and other contributors exploring camera-based physiological sensing and remote health-support concepts during COVID-19. The initiative was exploratory: it was not a controlled study and produced no published accuracy results.
VitalSign AI Platform Capabilities
- ·Pulse-rate estimation via RGB camera
- ·Higher-level state estimates (experimental; probabilistic)
- ·Posture tracking
- ·Alertness and fatigue indicators (experimental)
- ·API endpoints for integration
- ·Web-based research platform
How rPPG Works
Traditional photoplethysmography (PPG) uses contact-based sensors — typically pulse oximeters — to measure blood volume changes. Remote photoplethysmography extends this principle by using camera-captured video to detect the same micro-changes in skin reflectance.
An RGB camera captures facial video at standard frame rates. Signal processing algorithms isolate subtle color variations in regions of interest (typically the forehead and cheeks). These variations correspond to the cardiac cycle and respiratory patterns.
Machine learning models then filter noise — accounting for motion artifacts, lighting changes, and skin tone variation — to extract reliable physiological signals. Reported performance still depends on the specific algorithm, camera, lighting, motion, video compression, skin presentation and intended use.
Use Cases
Digital Health (possible application)
Remote monitoring and telehealth contexts are possible applications. They are not currently validated product capabilities and would require application-specific validation and regulatory review.
Adaptive UX Systems
Interfaces that respond in real time to measured signals — pulse-rate estimates, breathing patterns, movement and user feedback. Labels such as stress or fatigue are inferences drawn from those signals, not measurements.
Research Settings (possible application)
Physiological data collection for behavioural and interaction research, subject to consent, signal-quality limits and study-specific validation.
Human-Centered AI
AI systems that incorporate biological context into decision-making and personalization.
From Research to API
The path from academic exploration to commercial platform followed a deliberate arc. Early immersive systems at the University of Oregon (2014–2016) established the foundational understanding of human-state-aware environments.
Between 2017–2018, Hannes began integrating webcam-based physiological sensing into adaptive systems, exploring how rPPG signals could drive real-time environmental responses.
The 2020 international volunteer initiative with 170+ multidisciplinary contributors explored the feasibility of camera-based physiological sensing, informing the later architecture of VitalSign AI's API platform.
Signal versus inference
A camera records light reflected from skin. Signal processing turns that into a waveform, and the waveform supports a pulse-rate estimate. Everything after that — calm, stress, fatigue, attention — is a model’s guess about what the waveform means, carrying the assumptions of whoever trained it.
Camera-based estimation also degrades with motion, low or flickering light, video compression, and variation in skin presentation, and published evaluations report performance differences across skin types comparable to those documented for contact pulse oximeters. Signal quality should be surfaced to the user rather than hidden behind a confident-looking number, and no output here is diagnostic.
Frequently Asked Questions
What is remote photoplethysmography (rPPG)?
Remote photoplethysmography is a contactless method for estimating a pulse signal from standard RGB camera video, by detecting subtle changes in skin colour caused by blood-volume pulse variations. Pulse rate is the signal the method is best established for; breathing rate and any higher-level indicator derived from it are experimental and probabilistic.
How accurate is webcam-based pulse-rate estimation?
Published rPPG research has demonstrated accurate pulse-rate estimation under some controlled conditions. Performance depends on the specific algorithm, camera, lighting, motion, compression, skin presentation and intended use. Product-specific accuracy claims require application-specific validation.
What are the primary use cases for an rPPG API?
Explored and possible applications include adaptive user experience systems, research settings, wellbeing tools and human-centered AI that responds to physiological context. Health, telehealth, screening or fatigue-safety uses are possible applications only, and would require application-specific validation and appropriate regulatory review.
Does rPPG require special hardware?
No. rPPG works with standard RGB webcams and smartphone cameras. No wearable sensors, specialized lenses, or infrared hardware are required.