SFDA-rPPG: Source-Free Domain Adaptive Remote Physiological Measurement with Spatio-Temporal Consistency
School of Computing and Information Technology, Great Bay University（大湾区大学 计算与信息技术学院）；National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University（深圳大学 国家大数据系统计算技术工程实验室）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), Singapore（新加坡科技研究局高性能计算研究所）
Remote Photoplethysmography (rPPG) is a non-contact method that uses facial video to predict changes in blood volume, enabling physiological metrics measurement. Traditional rPPG models often struggle with poor generalization capacity in unseen domains. Current solutions to this problem is to improve its generalization in the target domain through Domain Generalization (DG) or Domain Adaptation (DA). However, both traditional methods require access to both source domain data and target domain data, which cannot be implemented in scenarios with limited access to source data, and another issue is the privacy of accessing source domain data. In this paper, we propose the first Source-free Domain Adaptation benchmark for rPPG measurement (SFDA-rPPG), which overcomes these limitations by enabling effective domain adaptation without access to source domain data. Our framework incorporates a Three-Branch Spatio-Temporal Consistency Network (TSTC-Net) to enhance feature consistency across domains. Furthermore, we propose a new rPPG distribution alignment loss based on the Frequency-domain Wasserstein Distance (FWD), which leverages optimal transport to align power spectrum distributions across domains effectively and further enforces the alignment of the three branches. Extensive cross-domain experiments and ablation studies demonstrate the effectiveness of our proposed method in source-free domain adaptation settings. Our findings highlight the significant contribution of the proposed FWD loss for distributional alignment, providing a valuable reference for future research and applications. The source code is available at https://github.com/XieYiping66/SFDA-rPPG
University of Oulu（奥卢大学）；University of Oxford（牛津大学）；Xi’an Jiaotong University（西安交通大学）；The University of Hong Kong（香港大学）
Remote photoplethysmography (rPPG), which aims at measuring heart activities and physiological signals from facial video without any contact, has great potential in many applications (e.g., remote healthcare and affective computing). Recent deep learning approaches focus on mining subtle rPPG clues using convolutional neural networks with limited spatio-temporal receptive fields, which neglect the long-range spatio-temporal perception and interaction for rPPG modeling. In this paper, we propose the PhysFormer, an end-to-end video transformer based architecture, to adaptively aggregate both loc…
Great Bay University（大湾区大学）；University of Oxford（牛津大学）；Xi’an Jiaotong University（西安交通大学）；The University of Hong Kong（香港大学）；University of Oulu（奥卢大学）
Remote photoplethysmography (rPPG), which aims at measuring heart activities and physiological signals from facial video without any contact, has great potential in many applications (e.g., remote healthcare and affective computing). Recent deep learning approaches focus on mining subtle rPPG clues using convolutional neural networks with limited spatio-temporal receptive fields, which neglect the long-range spatio-temporal perception and interaction for rPPG modeling. In this paper, we propose two end-to-end video transformer based architectures, namely PhysFormer and PhysFormer++, to adaptiv…
The Hong Kong University of Science & Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science & Technology（香港科技大学）；Great Bay University（大湾区大学）；Kuaishou Technology（快手科技）
Remote photoplethysmography (rPPG) technology has drawn increasing attention in recent years. It can extract Blood Volume Pulse (BVP) from facial videos, making many applications like health monitoring and emotional analysis more accessible. However, as the BVP signal is easily affected by environmental changes, existing methods struggle to generalize well for unseen domains. In this paper, we systematically address the domain shift problem in the rPPG measurement task. We show that most domain generalization methods do not work well in this problem, as domain labels are ambiguous in complicat…
DD-rPPGNet: De-interfering and Descriptive Feature Learning for Unsupervised rPPG Estimation
College of Computer and Cyber Security, Fujian Normal University（福建师范大学计算机与网络安全学院）；Department of Computer Science, National Tsing Hua University（国立清华大学计算机科学系）
HOT: Harmonic-Constrained Optimal Transport for Remote Photoplethysmography Domain Adaptation
VNU University of Engineering and Technology, Hanoi, Vietnam（越南河内国立大学工程与技术大学）；VinUni-Illinois Smart Health Center, VinUniversity, Hanoi, Vietnam（VinUni-Illinois智慧健康中心，VinUniversity，越南河内）；College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam（VinUniversity工程与计算机科学学院，越南河内）；Center for Innovations in Health Sciences, VinUniversity, Hanoi, Vietnam（VinUniversity健康科学创新中心，越南河内）
Self-similarity Prior Distillation for Unsupervised Remote Physiological Measurement
School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）；Artificial Intelligence Thrust, Hong Kong University of Science and Technology（香港科技大学（广州）人工智能学科）
Remote photoplethysmography (rPPG) enables non-contact heart rate (HR) estimation from facial videos which gives significant convenience compared with traditional contact-based measurements. In the real-world long-term health monitoring scenario, the distance of the participants and their head movements usually vary by time, resulting in the inaccurate rPPG measurement due to the varying face resolution and complex motion artifacts. Different from the previous rPPG models designed for a constant distance between camera and participants, in this paper, we propose two plug-and-play blocks (i.e.,…
Gaze into the Heart: A Multi-View Video Dataset for rPPG and Health Biomarkers Estimation
Sber AI Lab（Sber AI实验室）；Samara State Medical University（萨马拉州医学大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院信息与通信技术研究所可信人工智能研究中心）
Remote photoplethysmography (rPPG) allows for noncontact monitoring of blood volume changes from a camera by detecting minor fluctuations in reflected light. Prior applications of rPPG focused on face videos. In this paper we explored the feasibility of rPPG from non-face body regions such as the arms, legs, and hands. We collected a new dataset titled Multi-Site Physiological Monitoring (MSPM), which will be released with this paper. The dataset consists of 90 frames per second video of exposed arms, legs, and face, along with 10 synchronized PPG recordings. We performed baseline heart rate e…
Non-Contact Physiological Monitoring in Pediatric Intensive Care Units via Adaptive Masking and Self-Supervised Learning
Biomedical Information Processing Laboratory, École de Technologie Supérieure, University of Quebec（生物医学信息处理实验室，高等技术学院，魁北克大学）；Pediatric Intensive Care Unit, CHU Sainte-Justine（儿童重症监护单元，圣朱斯特医院）
MVRD-Bench: Multi-View Learning and Benchmarking for Dynamic Remote Photoplethysmography under Occlusion
Nanjing University of Information Science and Technology（南京信息工程大学）；University of Oulu（奥卢大学）；Xi’an Jiaotong University（西安交通大学）；Zhejiang University（浙江大学）
MHAD: Multimodal Home Activity Dataset with Multi-Angle Videos and Synchronized Physiological Signals
JD Health, Beijing, China（京东健康，北京，中国）；School of Software Engineering, Huazhong University of Science and Technology, Wuhan, China（华中科技大学软件工程学院，武汉，中国）；College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China（浙江大学生物医学工程与仪器科学学院，杭州，中国）
LQ-rPPG: A Label-Quantized Coarse-to-Fine Learning Framework for Remote Physiological Measurement
Electronics and Telecommunications Research Institute（电子电信研究院）；School of Electrical Engineering, Korea Advanced Institute of Science and Technology（韩国科学技术院电气工程学院）
Semi-rPPG: Semi-Supervised Remote Physiological Measurement with Curriculum Pseudo-Labeling
Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究所）；School of Computing and Information Technology, Great Bay University（大湾区大学计算与信息技术学院）；Computer Vision Institute, School of Computer Science & Software Engineering, Shenzhen University（深圳大学计算机科学与技术学院计算机视觉研究所）
rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement
Tianjin University（天津大学）；Lappeenranta-Lahti University of Technology LUT（拉彭兰塔-拉赫蒂工业大学（LUT大学））；Great Bay University（大湾区大学）；the Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
本文提出 rPPG-MAE 自监督预训练框架，通过线性重建任务和噪声不敏感策略挖掘生理信号的自相似先验，在多个 rPPG 数据集上超过现有自监督及 SOTA 监督方法。
PhysVR: Vision-Language Model Guided Interference-aware Temporal Feature Refinement for Remote Physiological Measurement
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；College of Computer Science and Technology, Qingdao University（青岛大学计算机科学技术学院）
Adaptive Physical-Facial Representation Fusion via Subject-Invariant Cross-Modal Prompt Tuning for Video-Based Emotion Recognition
Department of Biomedical Engineering（生物医学工程系）；Anhui Province Key Laboratory of Measuring Theory and Precision Instrument（安徽省测量理论与精密仪器重点实验室）；School of Computer Science and Information Engineering（计算机科学与信息工程学院）
FLOW: Feature-Level Optimal Warping for Generalized Remote Physiological Measurement
Great Bay University（大湾区大学）；Hefei University of Technology（合肥工业大学）；Southern University of Science and Technology（南方科技大学）；Macao Polytechnic University（澳门理工大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Dongguan Key Laboratory for Intelligence and Information Technology（东莞市智能与信息技术重点实验室）
FreqPhys: Repurposing Implicit Physiological Frequency Prior for Robust Remote Photoplethysmography
Hefei University of Technology（合肥工业大学）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；University of Science and Technology Beijing（北京科技大学）；Great Bay University（大湾区大学）
Northwest University（西北大学）；University of Oulu（奥卢大学）；Northwestern Polytechnical University（西北工业大学）
Automatic pain recognition is paramount for medical diagnosis and treatment. The existing works fall into three categories: assessing facial appearance changes, exploiting physiological cues, or fusing them in a multi-modal manner. However, (1) appearance changes are easily affected by subjective factors which impedes objective pain recognition. Besides, the appearance-based approaches ignore long-range spatial-temporal dependencies that are important for modeling expressions over time; (2) the physiological cues are obtained by attaching sensors on human body, which is inconvenient and uncomf…
PhysLLM: Harnessing Large Language Models for Cross-Modal Remote Physiological Sensing
Shenzhen University（深圳大学）；Great Bay University（大鹏大学）；National Engineering Laboratory for Big Data System Computing Technology（大数据系统计算技术国家工程实验室）；Dongguan Key Laboratory for Intelligence and Information Technology（东莞智能与信息技术重点实验室）；Guangdong Medical University（广东医科大学）；Southern Medical University (Dongguan People’s Hospital)（南方医科大学（东莞人民医院））；Macao Polytechnic University（澳门理工学院）
Robotics Institute（机器人研究所）；Carnegie Mellon University（卡内基梅隆大学）；Dept. of Computer Science（计算机科学系）；Binghamton University（宾汉姆顿大学）
Telehealth has the potential to offset the high demand for help during public health emergencies, such as the COVID-19 pandemic. Remote Photoplethysmography (rPPG) - the problem of non-invasively estimating blood volume variations in the microvascular tissue from video - would be well suited for these situations. Over the past few years a number of research groups have made rapid advances in remote PPG methods for estimating heart rate from digital video and obtained impressive results. How these various methods compare in naturalistic conditions, where spontaneous behavior, facial expressions…
Intervention-Based Self-Supervised Learning: A Causal Probe Paradigm for Remote Photoplethysmography
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601, PR China（合肥工业大学计算机科学与信息工程学院）；College of Aviation Electronic and Electrical Engineering, Civil Aviation Flight University of China, Guanghan 618307, PR China（中国民航飞行大学航空电子电气工程学院）；School of Computing and Information Technology, Great Bay University, Dongguan 523000, PR China（广东大湾区大学计算机科学与信息工程学院）
Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos
Hefei University of Technology（合肥工业大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；The 38th Research Institute of China Electronics Technology Group Corporation（中国电子科技集团公司第三十八研究所）
FCUS-rPPG: A Fast-Converging Unsupervised Framework for Remote Photoplethysmography via Gradient Oscillation Suppression
Department of Biomedical Engineering（生物医学工程系）；Anhui Province Key Laboratory of Measuring Theory and Precision Instrument（安徽省测量理论与精密仪器重点实验室）；Hefei University of Technology（合肥工业大学）；Department of Electronic Engineering and Information Science（电子工程与信息科学系）；University of Science and Technology of China（中国科学技术大学）
BTS-rPPG: Orthogonal Butterfly Temporal Shifting for Remote Photoplethysmography
VNU University of Engineering and Technology, Hanoi, Vietnam（越南河内国立大学工程与技术大学）；VinUni-Illinois Smart Health Center, VinUniversity, Hanoi, Vietnam（VinUni-Illinois智能健康中心，VinUniversity，越南河内）；College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam（VinUniversity工程与计算机科学学院，越南河内）；Center for Innovations in Health Sciences, VinUniversity, Hanoi, Vietnam（VinUniversity健康科学创新中心，越南河内）
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of Oulu（奥卢大学）；Peng Cheng Laboratory（鹏城实验室）
Remote physiological measurements, e.g., remote photoplethysmography (rPPG) based heart rate (HR), heart rate variability (HRV) and respiration frequency (RF) measuring, are playing more and more important roles under the application scenarios where contact measurement is inconvenient or impossible. Since the amplitude of the physiological signals is very small, they can be easily affected by head movements, lighting conditions, and sensor diversities. To address these challenges, we propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-phys…
Subtle periodic signals such as blood volume pulse and respiration can be extracted from RGB video, enabling remote health monitoring at low cost. Advancements in remote pulse estimation -- or remote photoplethysmography (rPPG) -- are currently driven by deep learning solutions. However, modern approaches are trained and evaluated on benchmark datasets with associated ground truth from contact-PPG sensors. We present the first non-contrastive unsupervised learning framework for signal regression to break free from the constraints of labelled video data. With minimal assumptions of periodicity …
