Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）；University of Oulu（奥卢大学）；Peng Cheng Laboratory（鹏城实验室）
Remote physiological measurements, e.g., remote photoplethysmography (rPPG) based heart rate (HR), heart rate variability (HRV) and respiration frequency (RF) measuring, are playing more and more important roles under the application scenarios where contact measurement is inconvenient or impossible. Since the amplitude of the physiological signals is very small, they can be easily affected by head movements, lighting conditions, and sensor diversities. To address these challenges, we propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations, and then use the distilled physiological features for robust multi-task physiological measurements. We first transform the input face videos into a multi-scale spatial-temporal map (MSTmap), which can suppress the irrelevant background and noise features while retaining most of the temporal characteristics of the periodic physiological signals. Then we take pairwise MSTmaps as inputs to an autoencoder architecture with two encoders (one for physiological signals and the other for non-physiological information) and use a cross-verified scheme to obtain physiological features disentangled with the non-physiological features. The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and rPPG signals. Comprehensive experiments on different large-scale public datasets of multiple physiological measurement tasks as well as the cross-database testing demonstrate the robustness of our approach.
University of Oulu（奥卢大学）；Institute of Computing Technology, CAS（中国科学院计算技术研究所）；INRIA（法国国家信息与自动化研究所）
Remote measurement of physiological signals from videos is an emerging topic. The topic draws great interests, but the lack of publicly available benchmark databases and a fair validation platform are hindering its further development. For this concern, we organize the first challenge on Remote Physiological Signal Sensing (RePSS), in which two databases of VIPL and OBF are provided as the benchmark for kin researchers to evaluate their approaches. The 1st challenge of RePSS focuses on measuring the average heart rate from facial videos, which is the basic problem of remote physiological measu…
Why Do VLMs Fail at Remote Physiological Measurement? Rethinking with Physiological Grounding and Numerical Alignment
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Sichuan University（四川大学）；Nanyang Technological University（南洋理工大学）；Tsinghua Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
FreqPhys: Repurposing Implicit Physiological Frequency Prior for Robust Remote Photoplethysmography
Hefei University of Technology（合肥工业大学）；Hefei Comprehensive National Science Center（合肥综合性国家科学中心）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）；University of Science and Technology Beijing（北京科技大学）；Great Bay University（大湾区大学）
Recent advances in supervised deep learning methods are enabling remote measurements of photoplethysmography-based physiological signals using facial videos. The performance of these supervised methods, however, are dependent on the availability of large labelled data. Contrastive learning as a self-supervised method has recently achieved state-of-the-art performances in learning representative data features by maximising mutual information between different augmented views. However, existing data augmentation techniques for contrastive learning are not designed to learn physiological signals …
Northwest University（西北大学）；University of Oulu（奥卢大学）；Northwestern Polytechnical University（西北工业大学）
Automatic pain recognition is paramount for medical diagnosis and treatment. The existing works fall into three categories: assessing facial appearance changes, exploiting physiological cues, or fusing them in a multi-modal manner. However, (1) appearance changes are easily affected by subjective factors which impedes objective pain recognition. Besides, the appearance-based approaches ignore long-range spatial-temporal dependencies that are important for modeling expressions over time; (2) the physiological cues are obtained by attaching sensors on human body, which is inconvenient and uncomf…
Disentangling Hardness from Noise: An Uncertainty-Driven Model-Agnostic Framework for Long-Tailed Remote Sensing Classification
Research Center for Space Computing System, Zhejiang Lab（空间计算系统研究中心，浙江实验室）；School of Cyber Science and Engineering, Zhengzhou University（网络安全工程学院，郑州大学）；School of Computer Science and Technology, Xi'an Jiaotong University（计算机科学与技术学院，西安交通大学）
FLOW: Feature-Level Optimal Warping for Generalized Remote Physiological Measurement
Great Bay University（大湾区大学）；Hefei University of Technology（合肥工业大学）；Southern University of Science and Technology（南方科技大学）；Macao Polytechnic University（澳门理工大学）；Harbin Institute of Technology, Shenzhen（哈尔滨工业大学（深圳））；Dongguan Key Laboratory for Intelligence and Information Technology（东莞市智能与信息技术重点实验室）
We present the Deception Detection and Physiological Monitoring (DDPM) dataset and initial baseline results on this dataset. Our application context is an interview scenario in which the interviewee attempts to deceive the interviewer on selected responses. The interviewee is recorded in RGB, near-infrared, and long-wave infrared, along with cardiac pulse, blood oxygenation, and audio. After collection, data were annotated for interviewer/interviewee, curated, ground-truthed, and organized into train / test parts for a set of canonical deception detection experiments. Baseline experiments foun…
PhysVR: Vision-Language Model Guided Interference-aware Temporal Feature Refinement for Remote Physiological Measurement
School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院）；College of Computer Science and Technology, Qingdao University（青岛大学计算机科学技术学院）
PhysLLM: Harnessing Large Language Models for Cross-Modal Remote Physiological Sensing
Shenzhen University（深圳大学）；Great Bay University（大鹏大学）；National Engineering Laboratory for Big Data System Computing Technology（大数据系统计算技术国家工程实验室）；Dongguan Key Laboratory for Intelligence and Information Technology（东莞智能与信息技术重点实验室）；Guangdong Medical University（广东医科大学）；Southern Medical University (Dongguan People’s Hospital)（南方医科大学（东莞人民医院））；Macao Polytechnic University（澳门理工学院）
The Hong Kong University of Science & Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science & Technology（香港科技大学）；Great Bay University（大湾区大学）；Kuaishou Technology（快手科技）
Remote photoplethysmography (rPPG) technology has drawn increasing attention in recent years. It can extract Blood Volume Pulse (BVP) from facial videos, making many applications like health monitoring and emotional analysis more accessible. However, as the BVP signal is easily affected by environmental changes, existing methods struggle to generalize well for unseen domains. In this paper, we systematically address the domain shift problem in the rPPG measurement task. We show that most domain generalization methods do not work well in this problem, as domain labels are ambiguous in complicat…
Subtle periodic signals such as blood volume pulse and respiration can be extracted from RGB video, enabling remote health monitoring at low cost. Advancements in remote pulse estimation -- or remote photoplethysmography (rPPG) -- are currently driven by deep learning solutions. However, modern approaches are trained and evaluated on benchmark datasets with associated ground truth from contact-PPG sensors. We present the first non-contrastive unsupervised learning framework for signal regression to break free from the constraints of labelled video data. With minimal assumptions of periodicity …
Bootstrapping Vision-language Models for Self-supervised Remote Physiological Measurement
College of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；Shanghai Institute of Intelligent Science and Technology, Tongji University（同济大学上海智能科学与技术研究院）；School of Management, Hefei University of Technology（合肥工业大学管理学院）
rPPG-MAE: Self-supervised Pre-training with Masked Autoencoders for Remote Physiological Measurement
Tianjin University（天津大学）；Lappeenranta-Lahti University of Technology LUT（拉彭兰塔-拉赫蒂工业大学（LUT大学））；Great Bay University（大湾区大学）；the Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））
本文提出 rPPG-MAE 自监督预训练框架，通过线性重建任务和噪声不敏感策略挖掘生理信号的自相似先验，在多个 rPPG 数据集上超过现有自监督及 SOTA 监督方法。
Semi-rPPG: Semi-Supervised Remote Physiological Measurement with Curriculum Pseudo-Labeling
Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR)（新加坡科技研究局高性能计算研究所）；School of Computing and Information Technology, Great Bay University（大湾区大学计算与信息技术学院）；Computer Vision Institute, School of Computer Science & Software Engineering, Shenzhen University（深圳大学计算机科学与技术学院计算机视觉研究所）
Not Only Consistency: Enhance Test-Time Adaptation with Spatio-temporal Inconsistency for Remote Physiological Measurement
Systems Hub, the Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）系统中心）；Information Hub, the Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）信息中心）；School of Information Engineering, Sichuan Agricultural University（四川农业大学信息工程学院）；School of Computing and Information Technology, Great Bay University（大湾大学计算与信息学院）
HiSem: Hierarchical Semantic Disentangling for Remote Sensing Image Change Captioning
College of Computer Science, Inner Mongolia University（内蒙古大学计算机学院）；Research Center for Spatiotemporal Intelligence, Inner Mongolia University（内蒙古大学时空智能研究中心）；Department of Aerospace Intelligent Science and Technology, School of Astronautics, Beihang University（北航宇航学院航空航天智能科学与技术系）；State Key Laboratory of Virtual Reality Technology and Systems, Beihang University（北航虚拟现实技术与系统国家重点实验室）；China Mobile Communications Group Inner Mongolia Co., Ltd.（中国移动通信集团内蒙古有限公司）
RPM-Distill: Physiology-guided Adaptive Cross-modal Distillation for Robust Remote Physiological Measurement
McGill University（麦吉尔大学）；Hong Kong University of Science and Technology（香港科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tsinghua University（清华大学）
Self-similarity Prior Distillation for Unsupervised Remote Physiological Measurement
School of Electronic Science and Engineering, Nanjing University（南京大学电子科学与工程学院）；Artificial Intelligence Thrust, Hong Kong University of Science and Technology（香港科技大学（广州）人工智能学科）
Remote photoplethysmography (rPPG) based physiological measurement has great application values in affective computing, non-contact health monitoring, telehealth monitoring, etc, which has become increasingly important especially during the COVID-19 pandemic. Existing methods are generally divided into two groups. The first focuses on mining the subtle blood volume pulse (BVP) signals from face videos, but seldom explicitly models the noises that dominate face video content. They are susceptible to the noises and may suffer from poor generalization ability in unseen scenarios. The second focus…
LQ-rPPG: A Label-Quantized Coarse-to-Fine Learning Framework for Remote Physiological Measurement
Electronics and Telecommunications Research Institute（电子电信研究院）；School of Electrical Engineering, Korea Advanced Institute of Science and Technology（韩国科学技术院电气工程学院）
Dual-path TokenLearner for Remote Photoplethysmography-based Physiological Measurement with Facial Videos
Hefei University of Technology（合肥工业大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；The 38th Research Institute of China Electronics Technology Group Corporation（中国电子科技集团公司第三十八研究所）
SFDA-rPPG: Source-Free Domain Adaptive Remote Physiological Measurement with Spatio-Temporal Consistency
School of Computing and Information Technology, Great Bay University（大湾区大学 计算与信息技术学院）；National Engineering Laboratory for Big Data System Computing Technology, Shenzhen University（深圳大学 国家大数据系统计算技术工程实验室）；Institute of High Performance Computing (IHPC), Agency for Science, Technology and Research (A*STAR), Singapore（新加坡科技研究局高性能计算研究所）
CardiacMamba: A Multimodal RGB-RF Fusion Framework with State Space Models for Remote Physiological Measurement
School of Computing and Information Technology, Great Bay University（湾大湾大学计算与信息技术学院）；Computer Vision Institute, School of Computer Science & Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院计算机视觉研究所）
SVC 2026: the Second Multimodal Deception Detection Challenge and the First Domain Generalized Remote Physiological Measurement Challenge
Wuhan University（武汉大学）；Great Bay University（大湾区大学）；Tsinghua University（清华大学）；The Chinese University of Hong Kong（香港中文大学）；Sun Yat-sen University（中山大学）；Hefei University of Technology（合肥工业大学）；University of Barcelona（巴塞罗那大学）
PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoring
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Tsinghua University（清华大学）；Sichuan Agricultural University（四川农业大学）
University of Washington（华盛顿大学）；Microsoft Research（微软研究院）；Fudan University（复旦大学）
The growing need for technology that supports remote healthcare is being acutely highlighted by an aging population and the COVID-19 pandemic. In health-related machine learning applications the ability to learn predictive models without data leaving a private device is attractive, especially when these data might contain features (e.g., photographs or videos of the body) that make identifying a subject trivial and/or the training data volume is large (e.g., uncompressed video). Camera-based remote physiological sensing facilitates scalable and low-cost measurement, but is a prime example of a…
VidFormer: A novel end-to-end framework fused by 3DCNN and Transformer for Video-based Remote Physiological Measurement
School of Information and Communication Engineering, University of Electronic Science and Technology of China（电子科技大学信息与通信工程学院）；Yangtze Delta Region Institute, University of Electronic Science and Technology of China（电子科技大学长三角研究院）
University of California, Los Angeles（加利福尼亚大学洛杉矶分校）
Camera-based remote photoplethysmography (rPPG) provides a non-contact way to measure physiological signals (e.g., heart rate) using facial videos. Recent deep learning architectures have improved the accuracy of such physiological measurement significantly, yet they are restricted by the diversity of the annotated videos. The existing datasets MMSE-HR, AFRL, and UBFC-RPPG contain roughly 10%, 0%, and 5% of dark-skinned subjects respectively. The unbalanced training sets result in a poor generalization capability to unseen subjects and lead to unwanted bias toward different demographic groups.…
University of Oulu（奥卢大学）；University of Oxford（牛津大学）；Xi’an Jiaotong University（西安交通大学）；The University of Hong Kong（香港大学）
Remote photoplethysmography (rPPG), which aims at measuring heart activities and physiological signals from facial video without any contact, has great potential in many applications (e.g., remote healthcare and affective computing). Recent deep learning approaches focus on mining subtle rPPG clues using convolutional neural networks with limited spatio-temporal receptive fields, which neglect the long-range spatio-temporal perception and interaction for rPPG modeling. In this paper, we propose the PhysFormer, an end-to-end video transformer based architecture, to adaptively aggregate both loc…
HOT: Harmonic-Constrained Optimal Transport for Remote Photoplethysmography Domain Adaptation
VNU University of Engineering and Technology, Hanoi, Vietnam（越南河内国立大学工程与技术大学）；VinUni-Illinois Smart Health Center, VinUniversity, Hanoi, Vietnam（VinUni-Illinois智慧健康中心，VinUniversity，越南河内）；College of Engineering and Computer Science, VinUniversity, Hanoi, Vietnam（VinUniversity工程与计算机科学学院，越南河内）；Center for Innovations in Health Sciences, VinUniversity, Hanoi, Vietnam（VinUniversity健康科学创新中心，越南河内）
Great Bay University（大湾区大学）；University of Oxford（牛津大学）；Xi’an Jiaotong University（西安交通大学）；The University of Hong Kong（香港大学）；University of Oulu（奥卢大学）
Remote photoplethysmography (rPPG), which aims at measuring heart activities and physiological signals from facial video without any contact, has great potential in many applications (e.g., remote healthcare and affective computing). Recent deep learning approaches focus on mining subtle rPPG clues using convolutional neural networks with limited spatio-temporal receptive fields, which neglect the long-range spatio-temporal perception and interaction for rPPG modeling. In this paper, we propose two end-to-end video transformer based architectures, namely PhysFormer and PhysFormer++, to adaptiv…
Intervention-Based Self-Supervised Learning: A Causal Probe Paradigm for Remote Photoplethysmography
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601, PR China（合肥工业大学计算机科学与信息工程学院）；College of Aviation Electronic and Electrical Engineering, Civil Aviation Flight University of China, Guanghan 618307, PR China（中国民航飞行大学航空电子电气工程学院）；School of Computing and Information Technology, Great Bay University, Dongguan 523000, PR China（广东大湾区大学计算机科学与信息工程学院）
