The Signal Is Not the Feeling

Remote photoplethysmography (rPPG) works from an ordinary camera. Blood volume changes with each heartbeat, skin reflectance changes minutely with blood volume, and a camera records that change as a periodic fluctuation across frames. From that fluctuation an algorithm estimates a pulse rate. Chest and shoulder motion, or the low-frequency component of the same trace, can yield a breathing-rate estimate.

That is the honest inventory: pulse-rate estimates, breathing patterns, movement, and whatever the person tells you directly. Not emotion. Not stress. Not attention. Those are inferences layered on top, and each layer adds error that the layer below cannot correct.

The 2026 roadmap by Elgendi and colleagues in npj Digital Medicine is blunt about where the field stands: heart rate is the one quantity with a credible path from camera to clinic, and even that path runs through standardised protocols, device-agnostic validation and regulatory work that mostly has not happened yet. Under favourable conditions — steady illumination, limited motion, a cooperative subject, a long-enough window — pulse-rate estimation can approach contact sensors. Respiration and heart-rate variability sit a step behind, and their reported accuracy depends heavily on the method and the validation set. Nothing in that literature licenses a jump to inferred psychological states.

The obvious worry is skin tone: melanin absorbs more of the light the method depends on, so the signal should be weaker on darker skin. The evidence is more specific than the worry. Dasari, Prakash, Jeni and Tucker (2021) ran the first large evaluation of bias in rPPG methods and reported no statistically significant country- or gender-associated difference in mean performance in their dataset. They concluded that skin tone might not have been the principal factor there. What did materially affect performance was the recording situation: motion, illumination, exposure and camera conditions, and the presence of multiple faces in frame.

That finding does not negate the demographic audit below, because the two studies ask different questions — one evaluates method performance under varying conditions, the other examines what the field’s datasets actually contain. The credible synthesis is that the evidence is mixed, and that performance depends on interactions among dataset composition, skin reflectance, illumination, motion, exposure and the method itself, rather than on any single demographic variable.

The datasets themselves are the second problem. Bondarenko, Menon and Elgendi (2025) audited the public rPPG datasets the field trains and benchmarks on and found them demographically narrow — skewed by skin tone, age and sex, and often too small to support subgroup claims at all. That study and the 2026 roadmap share two authors, Menon and Elgendi, so they should be read as a consistent programme of work rather than as independent corroboration.

Put together, the claim I am comfortable making is narrower than the usual one: rPPG is not proven to fail on dark skin, and it is not proven to work equitably either, because the data needed to establish that does not yet exist in public form. If you build on rPPG you own that uncertainty. The mitigations are unglamorous: report confidence, degrade to a neutral behaviour when confidence is low, evaluate per subgroup and per condition rather than in aggregate, and never let a low-confidence estimate drive a consequential decision.

Three distinct things get collapsed into one word, “detection”:

• Signal — a physical quantity recorded by a sensor. Reflectance change over time. This is measurable and falsifiable.
• Inference — a model’s probabilistic label placed on that signal: “elevated arousal”. This is a claim with an error rate, valid only for the population and conditions it was validated on.
• Meaning — what the state is about for the person living it. Grief and exhilaration can produce the same trace. Only the person knows which one it was, and they may not want to say.

Affective computing, as Rosalind Picard framed it, has always been careful about this boundary; the products built in its name frequently are not. The commercial temptation is to sell meaning at the price of signal.

My own work sits deliberately on the first rung. The systems described in the adaptive-interface patents adapt output — brightness, pacing, contrast, sound — in response to a measured signal. Adapting to a measurement is not a diagnosis, and I do not claim it is one. The interface can slow down when breathing gets shallow without ever asserting that it knows why.

Regulation (EU) 2024/1689 draws a line in roughly the same place. Among its prohibited practices is the placing on the market or use of AI systems to infer emotions of a natural person in the areas of workplace and education institutions, with a carve-out for medical or safety reasons. Emotion-inference systems outside those two settings are not banned — they fall into the transparency and, in several cases, high-risk obligations instead, including informing exposed people that such a system is in use. Biometric categorisation to deduce sensitive attributes is separately prohibited.

So the Act does not say “emotion AI is illegal”, and anyone who tells you it does is selling a simplification. What it does say is that in the two contexts where the power imbalance is starkest — your employer, your school — inferring your inner state from your body is off the table. That is a reasonable read of where inference stops being a feature and starts being coercion.

The practical consequence, in the systems I actually build, is a discipline about language before anything else. The interface names the signal and never the feeling: “breathing has been shallow for ten minutes” is checkable and can be contradicted; “you are anxious” is a verdict the person did not ask for and cannot appeal. Once you hold that line, most of the tempting features disappear on their own, because they only made sense as verdicts.

The second consequence is that the neutral behaviour gets designed first. An estimate arrives with a confidence, and below a threshold the system does nothing — no adaptation, no stored value, no quiet guess. Building the fallback first stops confidence from becoming a decoration bolted onto a system that was always going to act anyway. It also makes the failure mode boring, which is the correct failure mode for something pointed at a body.

The third is that the person outranks the model. If someone says the room has it wrong, the room is wrong, and there has to be somewhere in the interface for them to say it. And because the signal is worth more to an advertiser than it is to the person it came from, it stays local, it is not retained beyond the moment it is used, and it never crosses into targeting or assessment. That last one is a business decision disguised as an engineering one, which is why so few products get it right.

None of this makes the technology less useful. It makes it honest about what it is: a machine reading a body’s surface, offering the person a change they can accept or refuse. The signal is real. The feeling belongs to the person having it.