To date, research on sensor-equipped mobile devices has primarily focused on the purely supervised task of human activity recognition (walking, running, etc), demonstrating limited success in inferring high-level health outcomes from low-level signals, such as acceleration. Here, we present a novel self-supervised representation learning method using activity and heart rate (HR) signals without semantic labels. With a deep neural network, we set HR responses as the supervisory signal for the activity data, leveraging their underlying physiological relationship.
We evaluate our model in the largest free-living combined-sensing dataset (comprising more than 280,000 hours of wrist accelerometer & wearable ECG data) and show that the resulting embeddings can generalize in various downstream tasks through transfer learning with linear classifiers, capturing physiologically meaningful, personalized information. For instance, they can be used to predict (higher than 70 AUC) variables associated with individuals' health, fitness and demographic characteristics, outperforming unsupervised autoencoders and common bio-markers. Overall, we propose the first multimodal self-supervised method for behavioral and physiological data with implications for large-scale health and lifestyle monitoring.
University of North Carolina（北卡罗来纳大学）；George Washington University（乔治·华盛顿大学）
Breathing rate (BR), minute ventilation (VE), and other respiratory parameters are essential for real-time patient monitoring in many acute health conditions, such as asthma. The clinical standard for measuring respiration, namely Spirometry, is hardly suitable for continuous use. Wearables can track many physiological signals, like ECG and motion, yet not respiration. Deriving respiration from other modalities has become an area of active research. In this work, we infer respiratory parameters from wearable ECG and wrist motion signals. We propose a modular and generalizable classification-re…
Transformer-based Self-supervised Multimodal Representation Learning for Wearable Emotion Recognition
Univ. Lille（里尔大学）；CNRS（法国国家科学研究中心）；Centrale Lille（里尔中央理工学院）；Institut Mines-Télécom（法国国立高等矿业电信学校联盟）；IMT Nord Europe（法国国立高等矿业电信学校联盟北欧分校）；INSERM（法国国家健康与医学研究院）；University of Lille（里尔大学）
Wearable sensors provide abundant physiological time series, yet the principles governing their predictive utility remain unclear. We hypothesize that temporal resolution is a fundamental axis of representation learning, with different clinical and behavioral outcomes relying on structure at distinct scales. To test this resolution hypothesis, we introduce HiMAE (Hierarchical Masked Autoencoder), a self supervised framework that combines masked autoencoding with a hierarchical convolutional encoder decoder. HiMAE produces multi resolution embeddings that enable systematic evaluation of which t…
University of Cambridge（剑桥大学）；Alan Turing Institute（阿兰·图灵研究所）
Wearable devices such as smartwatches are becoming increasingly popular tools for objectively monitoring physical activity in free-living conditions. To date, research has primarily focused on the purely supervised task of human activity recognition, demonstrating limited success in inferring high-level health outcomes from low-level signals. Here, we present a novel self-supervised representation learning method using activity and heart rate (HR) signals without semantic labels. With a deep neural network, we set HR responses as the supervisory signal for the activity data, leveraging their u…
University of Massachusetts, Lowell（马萨诸塞大学洛厄尔分校）；University of Massachusetts Chan Medical School（马萨诸塞大学陈医学院）
Person re-identification is a critical privacy breach in publicly shared healthcare data. We investigate the possibility of a new type of privacy threat on publicly shared privacy insensitive large scale wearable sensing data. In this paper, we investigate user specific biometric signatures in terms of two contextual biometric traits, physiological (photoplethysmography and electrodermal activity) and physical (accelerometer) contexts. In this regard, we propose PhysioGait, a context-aware physiological signal model that consists of a Multi-Modal Siamese Convolutional Neural Network (mmSNN) wh…
To investigate whether a deep learning model can detect Covid-19 from disruptions in the human body's physiological (heart rate) and rest-activity rhythms (rhythmic dysregulation) caused by the SARS-CoV-2 virus. We propose CovidRhythm, a novel Gated Recurrent Unit (GRU) Network with Multi-Head Self-Attention (MHSA) that combines sensor and rhythmic features extracted from heart rate and activity (steps) data gathered passively using consumer-grade smart wearable to predict Covid-19. A total of 39 features were extracted (standard deviation, mean, min/max/avg length of sedentary and active bout…
Mobile proactive tourist recommender systems can support tourists by recommending the best choice depending on different contexts related to herself and the environment. In this paper, we propose to utilize wearable sensors to gather health information about a tourist and use them for recommending tourist activities. We discuss a range of wearable devices, sensors to infer physiological conditions of the users, and exemplify the feasibility using a popular self-quantification mobile app. Our main contribution then comprises a data model to derive relations between the parameters measured by th…
Towards Generalizable Drowsiness Monitoring with Physiological Sensors: A Preliminary Study
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；University of Toronto（多伦多大学）；The Hong Kong University of Science and Technology（香港科技大学）
Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals
Halıcıoğlu Data Science Institute, University of California San Diego（哈利奇奥格鲁数据科学研究所，加州大学圣地亚哥分校）；Schools of Engineering, Arizona State University（工程学院，亚利桑那州立大学）
Stress Detection Using Wearable Physiological and Sociometric Sensors
Dept. of Electronics, Computer Technology and Projects, Polytechnic University of Cartagena（电子工程与计算机技术项目系，卡塔赫纳理工大学）；Faculty of Automatic Control and Computer Science, Politehnica University of Bucharest（自动控制与计算机科学系，布加勒斯特理工大学）；Division of Psychology, Nottingham Trent University（心理学系，诺丁汉特伦特大学）；Department of Psychology, Lancaster University（心理学系，兰卡斯特大学）；School of Computer Science, University of Lincoln（计算机科学学院，林肯大学）
Physiological and behavioral data collected from wearable or mobile sensors have been used to estimate self-reported stress levels. Since the stress annotation usually relies on self-reports during the study, a limited amount of labeled data can be an obstacle in developing accurate and generalized stress predicting models. On the other hand, the sensors can continuously capture signals without annotations. This work investigates leveraging unlabeled wearable sensor data for stress detection in the wild. We first applied data augmentation techniques on the physiological and behavioral data to …
Health monitoring applications increasingly rely on machine learning techniques to learn end-user physiological and behavioral patterns in everyday settings. Considering the significant role of wearable devices in monitoring human body parameters, on-device learning can be utilized to build personalized models for behavioral and physiological patterns, and provide data privacy for users at the same time. However, resource constraints on most of these wearable devices prevent the ability to perform online learning on them. To address this issue, it is required to rethink the machine learning mo…
USC Information Sciences Institute（南加州大学信息科学研究所）；University of Washington（华盛顿大学）
Continuous collection of physiological data from wearable sensors enables temporal characterization of individual behaviors. Understanding the relation between an individual's behavioral patterns and psychological states can help identify strategies to improve quality of life. One challenge in analyzing physiological data is extracting the underlying behavioral states from the temporal sensor signals and interpreting them. Here, we use a non-parametric Bayesian approach to model sensor data from multiple people and discover the dynamic behaviors they share. We apply this method to data collect…
Physiological measurements involves observing variables that attribute to the normative functioning of human systems and subsystems directly or indirectly. The measurements can be used to detect affective states of a person with aims such as improving human-computer interactions. There are several methods of collecting physiological data, but wearable sensors are a common, non-invasive tool for accurate readings. However, valuable information is hard to extract from the raw physiological data, especially for affective state detection. Machine Learning techniques are used to detect the affectiv…
Are Anxiety Detection Models Generalizable? A Cross-Activity and Cross-Population Study Using Wearables
Indian Institute of Science Education and Research Bhopal (IISERB)（印度科学教育与研究学院博帕尔分校（IISERB））；All India Institute of Medical Sciences Bhopal（全印医学科学学院博帕尔分校）
Decoding Emotional Valence from Wearables: Can Our Data Reveal Our True Feelings?
Sano Centre for Computational Medicine（萨诺计算医学中心）；Poznan University of Technology（波兹南理工大学）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）
University of California Irvine（加州大学尔湾分校）；University of California, Irvine（加州大学尔湾分校）
Monitoring and understanding affective states are important aspects of healthy functioning and treatment of mood-based disorders. Recent advancements of ubiquitous wearable technologies have increased the reliability of such tools in detecting and accurately estimating mental states (e.g., mood, stress, etc.), offering comprehensive and continuous monitoring of individuals over time. Previous attempts to model an individual's mental state were limited to subjective approaches or the inclusion of only a few modalities (i.e., phone, watch). Thus, the goal of our study was to investigate the capa…
Towards Affordable, Non-Invasive Real-Time Hypoglycemia Detection Using Wearable Sensor Signals
Department of Computer Science, Old Dominion University, Norfolk, VA, USA（计算机科学系，旧 Dominion 大学）；Virginia Digital Maritime Center (VDMC), Old Dominion University, Norfolk, VA, USA（弗吉尼亚数字水路中心（VDMC），旧 Dominion 大学）
Comprehensive Signal Quality Evaluation of a Wearable Textile ECG Garment: A Sex-Balanced Study
Department Digital Health and Analytics, Fraunhofer IIS, Fraunhofer Institute for Integrated Circuits IIS（数字健康与分析部门，弗劳恩霍夫研究所IIS，弗劳恩霍夫集成电路研究所）；Department Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-University Erlangen Nuremberg（生物医学工程中的人工智能部门，埃尔兰根-纽伦堡弗赖堡-亚历山大大学）
The advent of IoT has enabled the design of connected and integrated smart health monitoring systems. These smart health monitoring systems could be realized in a smart home context to render long-term care to the elderly population. In this paper, we present the design of a wearable health monitoring system suitable for older adults in a smart home context. The proposed system offers solutions to monitor the stress, blood pressure, and location of an individual within a smart home environment. The stress detection model proposed in this work uses Electrodermal Activity (EDA), Photoplethysmogr…
Wearable devices enable theoretically continuous, longitudinal monitoring of physiological measurements like step count, energy expenditure, and heart rate. Although the classification of abnormal cardiac rhythms such as atrial fibrillation from wearable devices has great potential, commercial algorithms remain proprietary and tend to focus on heart rate variability derived from green spectrum LED sensors placed on the wrist where noise remains an unsolved problem. Here, we develop a multi-task deep learning method to assess signal quality and arrhythmia event detection in wearable photoplethy…
WearableMil: An End-to-End Framework for Military Activity Recognition and Performance Monitoring
Department of Computer Science（计算机科学系）；Technion—Israel Institute of Technology（技术ion—以色列理工学院）；Department of Information Systems（信息系统系）；University of Haifa（海法大学）
Eindhoven University of Technology（埃因霍温理工大学）；Philips Research（飞利浦研究院）
Stress can be seen as a physiological response to everyday emotional, mental and physical challenges. A long-term exposure to stressful situations can have negative health consequences, such as increased risk of cardiovascular diseases and immune system disorder. Therefore, a timely stress detection can lead to systems for better management and prevention in future circumstances. In this paper, we suggest a multi-task learning based neural network approach (with hard parameter sharing of mutual representation and task-specific layers) for personalized stress recognition using skin conductance …
HealthLoopQA: A Context-Aware Question Answering Benchmark for Interpreting Wearable Monitoring Data in Diabetes Care
Mayo Clinic（梅奥诊所）；Imperial College London（伦敦帝国学院）；Imperial Global Singapore（新加坡帝国全球学院）；National University of Singapore（新加坡国立大学）；The University of Manchester（曼彻斯特大学）；Nanyang Technological University（南洋理工大学）
Cognitive fatigue has been a common problem among workers which has become an increasing global problem since the emergence of COVID-19 as a global pandemic. While existing multi-modal wearable sensors-aided automatic cognitive fatigue monitoring tools have focused on physical and physiological sensors (ECG, PPG, Actigraphy) analytic on specific group of people (say gamers, athletes, construction workers), activity-awareness is utmost importance due to its different responses on physiology in different person. In this paper, we propose a novel framework, Activity-Aware Recurrent Neural Network…
Electrodermal Activity as a Unimodal Signal for Aerobic Exercise Detection in Wearable Sensors
Odle Middle School（Odle中学）；ImagineQ Labs（ImagineQ实验室）；Cambridge Center for International（剑桥国际中心）；bellevue, WA, USA（西雅图，华盛顿州，美国）；Research, United Kingdom（英国研究）
The growing popularity of wearable sensors has generated large quantities of temporal physiological and activity data. Ability to analyze this data offers new opportunities for real-time health monitoring and forecasting. However, temporal physiological data presents many analytic challenges: the data is noisy, contains many missing values, and each series has a different length. Most methods proposed for time series analysis and classification do not handle datasets with these characteristics nor do they offer interpretability and explainability, a critical requirement in the health domain. W…
Low-power sensing technologies, such as wearables, have emerged in the healthcare domain since they enable continuous and non-invasive monitoring of physiological signals. In order to endow such devices with clinical value, classical signal processing has encountered numerous challenges. However, data-driven methods, such as machine learning, offer attractive accuracies at the expense of being resource and memory demanding. In this paper, we focus on the inference of neural networks running in microcontrollers and low-power processors which wearable sensors and devices are generally equipped w…
RMIT University（皇家墨尔本理工大学）；Swinburne University of Technology（斯威本科技大学）；University of New South Wales (UNSW)（新南威尔士大学（UNSW））
We conducted a field study at a K-12 private school in the suburbs of Melbourne, Australia. The data capture contained two elements: First, a 5-month longitudinal field study In-Gauge using two outdoor weather stations, as well as indoor weather stations in 17 classrooms and temperature sensors on the vents of occupant-controlled room air-conditioners; these were collated into individual datasets for each classroom at a 5-minute logging frequency, including additional data on occupant presence. The dataset was used to derive predictive models of how occupants operate room air-conditioning unit…
With the development of wearable technologies, a new kind of healthcare data has become valuable as medical information. These data provide meaningful information regarding an individual's physiological and psychological states, such as activity level, mood, stress, and cognitive health. These biomarkers are named digital since they are collected from digital devices integrated with various sensors. In this study, we explore digital biomarkers related to stress modality by examining data collected from mobile phones and smartwatches. We utilize machine learning techniques on the Tesserae datas…
With the advancement of technologies, market wearables are becoming increasingly popular with a range of services, including providing access to bank accounts, accessing cars, monitoring patients remotely, among several others. However, often these wearables collect various sensitive personal information of a user with no to limited authentication, e.g., knowledge-based external authentication techniques, such as PINs. While most of these external authentication techniques suffer from multiple limitations, including recall burden, human errors, or biases, researchers have started using various…
