The prevalence of wearable sensors (e.g., smart wristband) is creating unprecedented opportunities to not only inform health and wellness states of individuals, but also assess and infer personal attributes, including demographic and personality attributes. However, the data captured from wearables, such as heart rate or number of steps, present two key challenges: 1) the time series is often of variable-length and incomplete due to different data collection periods (e.g., wearing behavior varies by person); and 2) inter-individual variability to external factors like stress and environment. This paper addresses these challenges and brings us closer to the potential of personalized insights about an individual, taking the leap from quantified self to qualified self. Specifically, HeartSpace proposed in this paper encodes time series data with variable-length and missing values via the integration of a time series encoding module and a pattern aggregation network. Additionally, HeartSpace implements a Siamese-triplet network to optimize representations by jointly capturing intra- and inter-series correlations during the embedding learning process. The empirical evaluation over two different real-world data presents significant performance gains overstate-of-the-art baselines in a variety of applications, including personality prediction, demographics inference, and user identification.
Queen Mary University of London（伦敦玛丽女王大学）；Imperial College London（帝国理工学院）
An increasing number of sensors on mobile, Internet of things (IoT), and wearable devices generate time-series measurements of physical activities. Though access to the sensory data is critical to the success of many beneficial applications such as health monitoring or activity recognition, a wide range of potentially sensitive information about the individuals can also be discovered through access to sensory data and this cannot easily be protected using traditional privacy approaches. In this paper, we propose a privacy-preserving sensing framework for managing access to time-series data in …
University of Washington（华盛顿大学）；University of Texas at Austin（德克萨斯大学奥斯汀分校）
In wearable sensing applications, data is inevitable to be irregularly sampled or partially missing, which pose challenges for any downstream application. An unique aspect of wearable data is that it is time-series data and each channel can be correlated to another one, such as x, y, z axis of accelerometer. We argue that traditional methods have rarely made use of both times-series dynamics of the data as well as the relatedness of the features from different sensors. We propose a model, termed as DynImp, to handle different time point's missingness with nearest neighbors along feature axis a…
Automatic recognition of human activities from time-series sensor data (referred to as HAR) is a growing area of research in ubiquitous computing. Most recent research in the field adopts supervised deep learning paradigms to automate extraction of intrinsic features from raw signal inputs and addresses HAR as a multi-class classification problem where detecting a single activity class within the duration of a sensory data segment suffices. However, due to the innate diversity of human activities and their corresponding duration, no data segment is guaranteed to contain sensor recordings of a …
Intelligent upper-limb exoskeleton integrated with soft wearable bioelectronics and deep-learning for human intention-driven strength augmentation based on sensory feedback
Dongguk University（东国大学）；Georgia Institute of Technology（佐治亚理工学院）；Yonsei University（延世大学）；Emory University School of Medicine（埃默里大学医学院）
Our ability to exploit low-cost wearable sensing modalities for critical human behaviour and activity monitoring applications in health and wellness is reliant on supervised learning regimes; here, deep learning paradigms have proven extremely successful in learning activity representations from annotated data. However, the costly work of gathering and annotating sensory activity datasets is labor-intensive, time consuming and not scalable to large volumes of data. While existing unsupervised remedies of deep clustering leverage network architectures and optimization objectives that are tailor…
MoCA: Multi-modal Cross-masked Autoencoder for Time Series in Digital Health
Herbert Wertheim School of Public Health and Human Longevity Science, University of California, San Diego, San Diego, CA, USA（赫伯特·韦瑟姆公共卫生与人类长寿科学学院，加州大学圣地亚哥分校）；Halıcıoğlu Data Science Institute, University of California San Diego, San Diego, CA, USA（哈利奇奥卢数据科学研究所，加州大学圣地亚哥分校）；Department of Computer Science and Engineering, University of California San Diego, San Diego, CA, USA（计算机科学与工程系，加州大学圣地亚哥分校）；Division of Public Health Sciences, Fred Hutchinson Cancer Center, Seattle, WA, USA（公共卫生科学部，Fred Hutchinson癌症中心）
MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；Google Research（谷歌研究院）
Humans view the world through many sensory channels, e.g., the long-wavelength light channel, viewed by the left eye, or the high-frequency vibrations channel, heard by the right ear. Each view is noisy and incomplete, but important factors, such as physics, geometry, and semantics, tend to be shared between all views (e.g., a "dog" can be seen, heard, and felt). We investigate the classic hypothesis that a powerful representation is one that models view-invariant factors. We study this hypothesis under the framework of multiview contrastive learning, where we learn a representation that aims …
SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals
Georgia Institute of Technology（佐治亚理工学院）；Duke University（杜克大学）；Emory University（埃默里大学）；University of California at San Francisco（加州大学旧金山分校）；Nell Hodgson Woodruff School of Nursing, Emory University（埃默里大学内尔·霍奇森·伍德拉夫护理学院）
Novel Smart N95 Filtering Facepiece Respirator with Real-time Adaptive Fit Functionality and Wireless Humidity Monitoring for Enhanced Wearable Comfort
Disentangling Imperfect: A Wavelet-Infused Multilevel Heterogeneous Network for Human Activity Recognition in Flawed Wearable Sensor Data
School of Computer Science and Engineering, Tianjin University of Technology（天津理工大学计算机科学与工程学院）；Technical University of Denmark（丹麦技术大学）；Aalborg University（奥尔堡大学）
New York University（纽约大学）；New York University School of Medicine（纽约大学医学院）；Courant Institute of Mathematical Sciences（库朗数学科学研究所）
Recovery after stroke is often incomplete, but rehabilitation training may potentiate recovery by engaging endogenous neuroplasticity. In preclinical models of stroke, high doses of rehabilitation training are required to restore functional movement to the affected limbs of animals. In humans, however, the necessary dose of training to potentiate recovery is not known. This ignorance stems from the lack of objective, pragmatic approaches for measuring training doses in rehabilitation activities. Here, to develop a measurement approach, we took the critical first step of automatically identifyi…
Multimodal wearable physiological data in daily life have been used to estimate self-reported stress labels. However, missing data modalities in data collection makes it challenging to leverage all the collected samples. Besides, heterogeneous sensor data and labels among individuals add challenges in building robust stress detection models. In this paper, we proposed a modality fusion network (MFN) to train models and infer self-reported binary stress labels under both complete and incomplete modality conditions. In addition, we applied personalized attention (PA) strategy to leverage persona…
