From Immersive Art to Remote Photoplethysmography: The Evolution of Bioadaptive AI

1. Early Immersive Systems (2014–2016)

This work did not start in a computer science lab. It started between an art practice, a quantum and nanoscale physics laboratory, and contemplative neuroscience at the University of Oregon. "Metaverses / Seecular" (2014–2016) ran across the Alemán Lab, cognitive neuroscience researchers, and my own studio work on perception.

The question was simple enough to state and hard to answer: could a digital environment respond to a human state rather than to a click? Could a system tell — without being told — whether someone was focused, agitated or calm, and change what it showed?

The tools were crude. EEG headsets gave coarse signals; the audiovisual environments were hand-built, scene by scene. But the premise held: adapt to biology, not to behaviour.

2. Neuroscience Collaborations

The University of Oregon provided a unique research environment. Collaboration with Dr. Michael Posner, one of the most cited cognitive neuroscientists in history, grounded the artistic intuitions in rigorous scientific methodology. Posner's work on attention networks provided a theoretical framework for understanding how environmental stimuli affect cognitive states.

Edward Vogel's laboratory, specializing in attention and working memory, offered the technical infrastructure for measuring neural responses to visual stimuli. The EEG study "Correlation between Visual Stimuli and Brain States" produced quantifiable data connecting specific visual patterns to measurable changes in brain activity.

3. EEG and fMRI Integration

The research extended beyond EEG into functional magnetic resonance imaging. The fMRI study "Neural Mechanisms of Multiple Meditation Techniques within Practitioners," conducted at the Lewis Center for Neuroimaging, examined how different contemplative practices produced distinct patterns of neural activation.

This dual-modality approach — combining the temporal resolution of EEG with the spatial resolution of fMRI — provided a more complete picture of how human states manifest neurologically. The implications for interface design were clear: if states could be measured, they could potentially be influenced through carefully designed environmental responses.

4. Transition to Physiological Computing

The presentation at the Association for the Advancement of Artificial Intelligence (AAAI) Spring Symposium 2016 marked a pivotal transition. The paper "Mindful Technologies: Research and Developments in Science and Art" — which received the Best Presentation Award — articulated for the first time a coherent vision of technology that incorporates contemplative principles into computational systems.

Simultaneously, the Science of Consciousness 2016 conference at the University of Arizona provided a platform for presenting the biofeedback visualization system to the consciousness studies community. These dual presentations — one to the AI community, one to the consciousness research community — positioned the work at the intersection of both fields.

5. Remote Photoplethysmography Explained

Between 2017 and 2018, the research shifted from laboratory-grade sensing equipment to commodity hardware. The key insight was that standard RGB webcams could detect physiological signals that previously required specialized medical devices.

Remote photoplethysmography (rPPG) exploits the fact that blood volume changes in subcutaneous tissue produce micro-variations in skin color. These variations are invisible to the human eye but detectable by camera sensors operating at standard frame rates. Signal processing algorithms extract the cardiac pulse wave from facial video, enabling contactless measurement of heart rate, heart rate variability, and respiratory patterns.

That changes who can be measured. No wearable, no clinic visit, no appointment — any device with a camera becomes a possible sensor, which is both the opportunity and the reason to be careful about it.

6. The COVID-19 Pilot Initiative

The 2020 COVID-19 pandemic created both urgency and opportunity for remote physiological sensing. With populations isolated and healthcare systems overwhelmed, the need for contactless health monitoring became acute.

In 2020, Hannes Bend led an international volunteer initiative involving more than 170 healthcare practitioners, researchers, data scientists, designers, developers and other contributors exploring camera-based physiological sensing and remote health-support concepts during COVID-19.

The initiative was exploratory rather than a controlled study: it produced no published accuracy results and no participant-level validation dataset. Its main outcome was conceptual and architectural, informing the later design of VitalSign AI's platform.

7. Commercial API Systems

The transition from research prototype to commercial product required fundamental architectural decisions. VitalSign AI's API platform provides standardized endpoints for heart rate detection, posture tracking and experimental alertness estimates through standard webcam input. Higher-level state estimates may combine pulse-related signals with posture, facial behavior, interaction patterns and context; such estimates are probabilistic and require appropriate validation.

Complementary platforms extend the bioadaptive vision: Breathing AI provides an adaptive browser extension that modifies screen content based on detected user states. Optimizing AI focuses on performance optimization through physiological awareness. Breathing AI reports users in more than 150 countries according to its product records.

8. The Future of Adaptive AI

Cameras are already everywhere and the models keep getting cheaper to run, so sensing human state will stop being a feature and become an assumption. That is precisely why it matters who sets the defaults now.

The near work is multi-modal — visual, audio and behavioural signals read together — processed at the edge, with adaptation models that learn one person's baseline instead of averaging everyone into a population.

9. Ethical Implications

Systems that sense biological state carry profound ethical responsibilities. The same technology that enables wellbeing-focused adaptation could be misused for surveillance, manipulation, or discriminatory profiling.

Hannes Bend's approach has consistently prioritized several ethical principles: on-device processing (biological data should not leave the user's device), informed consent (users must understand what is being sensed), minimal retention (physiological data should be processed and discarded, not stored), and user sovereignty (the user, not the platform, controls how their biological data influences their experience).

These principles guide the architecture and continuing development of VitalSign AI and Breathing AI. Their implementation depends on the specific product, processing configuration and deployment, and should be documented and verified for each use case.

10. Conclusion

The line from immersive art to remote photoplethysmography is not a career story. It is a change in what a machine is listening to: from what we type and tap, to what our bodies are already doing.

An inquiry that began in a physics lab and an artist's studio in Oregon now runs through granted patents, peer-reviewed presentations and products people use daily. What keeps it coherent is the target: not engagement, not time on screen, but whether the person in front of the system is doing better with it than without it.