MHAD: Multimodal Home Activity Dataset with Multi-Angle Videos and Synchronized Physiological Signals
JD Health, Beijing, China（京东健康，北京，中国）；School of Software Engineering, Huazhong University of Science and Technology, Wuhan, China（华中科技大学软件工程学院，武汉，中国）；College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, China（浙江大学生物医学工程与仪器科学学院，杭州，中国）
Lei Yu, Jintao Fei, Xinyi Liu, Yang Yao, Jun Zhao, Guoxin Wang, Xin Li
Video-based physiology, exemplified by remote photoplethysmography (rPPG), extracts physiological signals such as pulse and respiration by analyzing subtle changes in video recordings. This non-contact, real-time monitoring method holds great potential for home settings. Despite the valuable contributions of public benchmark datasets to this technology, there is currently no dataset specifically designed for passive home monitoring. Existing datasets are often limited to close-up, static, frontal recordings and typically include only 1-2 physiological signals. To advance video-based physiology in real home settings, we introduce the MHAD dataset. It comprises 1,440 videos from 40 subjects, capturing 6 typical activities from 3 angles in a real home environment. Additionally, 5 physiological signals were recorded, making it a comprehensive video-based physiology dataset. MHAD is compatible with the rPPG-toolbox and has been validated using several unsupervised and supervised methods. Our dataset is publicly available at https://github.com/jdh-algo/MHAD-Dataset.
Human action recognition has been widely used in many fields of life, and many human action datasets have been published at the same time. However, most of the multi-modal databases have some shortcomings in the layout and number of sensors, which cannot fully represent the action features. Regarding the problems, this paper proposes a freely available dataset, named CZU-MHAD (Changzhou University: a comprehensive multi-modal human action dataset). It consists of 22 actions and three modals temporal synchronized data. These modals include depth videos and skeleton positions from a kinect v2 ca…
PhysDrive: A Multimodal Remote Physiological Measurement Dataset for In-vehicle Driver Monitoring
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Tsinghua University（清华大学）；Sichuan Agricultural University（四川农业大学）
RPM-Distill: Physiology-guided Adaptive Cross-modal Distillation for Robust Remote Physiological Measurement
McGill University（麦吉尔大学）；Hong Kong University of Science and Technology（香港科技大学）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tsinghua University（清华大学）
Gaze into the Heart: A Multi-View Video Dataset for rPPG and Health Biomarkers Estimation
Sber AI Lab（Sber AI实验室）；Samara State Medical University（萨马拉州医学大学）；ISP RAS Research Center for Trusted Artificial Intelligence（俄罗斯科学院信息与通信技术研究所可信人工智能研究中心）
Remote photoplethysmography (rPPG) allows for noncontact monitoring of blood volume changes from a camera by detecting minor fluctuations in reflected light. Prior applications of rPPG focused on face videos. In this paper we explored the feasibility of rPPG from non-face body regions such as the arms, legs, and hands. We collected a new dataset titled Multi-Site Physiological Monitoring (MSPM), which will be released with this paper. The dataset consists of 90 frames per second video of exposed arms, legs, and face, along with 10 synchronized PPG recordings. We performed baseline heart rate e…
M3PD Dataset: Dual-view Photoplethysmography (PPG) Using Front-and-rear Cameras of Smartphones in Lab and Clinical Settings
Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术系）；Beijing Anzhen Hospital, Capital Medical University（首都医科大学北京安贞医院）；Agency for Science, Technology and Research（科技研究局）
EgoIntrospect: An Egocentric Dataset and Benchmark for User-Centric Internal State Reasoning
Tsinghua University（清华大学）；Tongji University（同济大学）；Renmin University of China（中国人民大学）；The University of Tokyo（东京大学）；Lenovo Group（联想集团）；Peking University（北京大学）；University of Wisconsin–Madison（威斯康星大学麦迪逊分校）；Shanghai Qi Zhi Institute（上海启智研究院）
MVRD-Bench: Multi-View Learning and Benchmarking for Dynamic Remote Photoplethysmography under Occlusion
Nanjing University of Information Science and Technology（南京信息工程大学）；University of Oulu（奥卢大学）；Xi’an Jiaotong University（西安交通大学）；Zhejiang University（浙江大学）
EduGage: A Multimodal Dataset and Benchmark for Sensor-Based Momentary Assessment of Engagement in Self-Guided Video Learning
School of Interactive Computing, Georgia Institute of Technology（佐治亚理工学院交互计算学院）；College of Computing, Georgia Institute of Technology（佐治亚理工学院计算机学院）；School of Computing Technologies, RMIT University（皇家墨尔本理工大学计算技术学院）
FLEX: A Largescale Multimodal, Multiview Dataset for Learning Structured Representations for Fitness Action Quality Assessment
School of Biomedical Engineering (Suzhou), USTC（中国科学技术大学苏州生物医学工程学院）；Suzhou Institute of Biomedical Engineering and Technology, CAS（中国科学院苏州生物医学工程技术研究所）；Institute of High-Performance Computing, A*STAR, Singapore（新加坡科技研究局高性能计算研究所）；School of Psychology, Beijing Sports University（北京体育大学心理学院）；School of Competitive Sports, Beijing Sports University（北京体育大学竞技体育学院）；Department of Robotics, University of Michigan（密歇根大学机器人系）
MyoMechanix: Biomechanically-Grounded Compositional Skilled Activity Understanding and Coaching
University of Science and Technology of China（中国科学技术大学）；Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences（中国科学院苏州生物医学工程技术研究所）；Arexeni Research and Technologies Inc.（Arexeni研究与技术公司）；Beijing Sports University（北京体育大学）；University of Michigan（密歇根大学）
On-Device Inference versus Wireless Streaming: Energy-Efficient Multi-Modal Deep Learning for Wearable Cardiovascular Patches
CTO System Innovation, NXP Semiconductors Germany GmbH（NXP半导体德国系统创新部）；Advanced Chip Engineering, NXP Semiconductors（NXP半导体先进芯片工程部）；Business Line Secure Connected Edge, NXP Semiconductors（NXP半导体安全连接边缘业务线）；Institute of Medical Technology and Intelligent Systems, Hamburg University of Technology（汉堡技术大学医学技术与智能系统研究所）；Department of Informatics, Hamburg University of Applied Sciences（汉堡应用科学大学信息学院）
