Wearable sensor-based human activity recognition (HAR) has emerged as a principal research area and is utilized in a variety of applications. Recently, deep learning-based methods have achieved significant improvement in the HAR field with the development of human-computer interaction applications. However, they are limited to operating in a local neighborhood in the process of a standard convolution neural network, and correlations between different sensors on body positions are ignored. In addition, they still face significant challenging problems with performance degradation due to large gaps in the distribution of training and test data, and behavioral differences between subjects. In this work, we propose a novel Transformer-based Adversarial learning framework for human activity recognition using wearable sensors via Self-KnowledgE Distillation (TASKED), that accounts for individual sensor orientations and spatial and temporal features. The proposed method is capable of learning cross-domain embedding feature representations from multiple subjects datasets using adversarial learning and the maximum mean discrepancy (MMD) regularization to align the data distribution over multiple domains. In the proposed method, we adopt the teacher-free self-knowledge distillation to improve the stability of the training procedure and the performance of human activity recognition. Experimental results show that TASKED not only outperforms state-of-the-art methods on the four real-world public HAR datasets (alone or combined) but also improves the subject generalization effectively.
Integrating low-power wearable Internet of Things (IoT) systems into routine health monitoring is an ongoing challenge. Recent advances in the computation capabilities of wearables make it possible to target complex scenarios by exploiting multiple biosignals and using high-performance algorithms, such as Deep Neural Networks (DNNs). There is, however, a trade-off between performance of the algorithms and the low-power requirements of IoT platforms with limited resources. Besides, physically larger and multi-biosignal-based wearables bring significant discomfort to the patients. Consequently, …
Sun Yat-Sen University（中山大学）；DMAI Co., Ltd（DMAI有限公司）
Existing vision-based action recognition is susceptible to occlusion and appearance variations, while wearable sensors can alleviate these challenges by capturing human motion with one-dimensional time-series signal. For the same action, the knowledge learned from vision sensors and wearable sensors, may be related and complementary. However, there exists significantly large modality difference between action data captured by wearable-sensor and vision-sensor in data dimension, data distribution and inherent information content. In this paper, we propose a novel framework, named Semantics-awar…
Arizona State University（亚利桑那州立大学）；SRI International（斯坦福国际研究院）
Deep neural networks are parametrized by several thousands or millions of parameters, and have shown tremendous success in many classification problems. However, the large number of parameters makes it difficult to integrate these models into edge devices such as smartphones and wearable devices. To address this problem, knowledge distillation (KD) has been widely employed, that uses a pre-trained high capacity network to train a much smaller network, suitable for edge devices. In this paper, for the first time, we study the applicability and challenges of using KD for time-series data for wea…
Texas State University（德克萨斯州立大学）；Illinois Institute of Technology（伊利诺伊理工学院）
Wearable sensor-based Human Action Recognition (HAR) has achieved remarkable success recently. However, the accuracy performance of wearable sensor-based HAR is still far behind the ones from the visual modalities-based system (i.e., RGB video, skeleton, and depth). Diverse input modalities can provide complementary cues and thus improve the accuracy performance of HAR, but how to take advantage of multi-modal data on wearable sensor-based HAR has rarely been explored. Currently, wearable devices, i.e., smartwatches, can only capture limited kinds of non-visual modality data. This hinders the …
PPG-Distill: Efficient Photoplethysmography Signals Analysis via Foundation Model Distillation
Department of Computer Science, Emory University（计算机科学系，埃默里大学）；Nell Hodgson Woodruff School of Nursing, Emory University（Nell Hodgson Woodruff护理学院，埃默里大学）
Texas State University（德克萨斯州立大学）；University of Houston（休斯顿大学）；Sun-Yat-Sen University（中山大学）；Illinois Institute of Technology（伊利诺伊理工学院）
Human activity recognition (HAR) based on multi-modal approach has been recently shown to improve the accuracy performance of HAR. However, restricted computational resources associated with wearable devices, i.e., smartwatch, failed to directly support such advanced methods. To tackle this issue, this study introduces an end-to-end Vision-to-Sensor Knowledge Distillation (VSKD) framework. In this VSKD framework, only time-series data, i.e., accelerometer data, is needed from wearable devices during the testing phase. Therefore, this framework will not only reduce the computational demands on …
Role of Mixup in Topological Persistence Based Knowledge Distillation for Wearable Sensor Data
Seoul National University of Science and Technology（首尔科学技术大学）；Lawrence Livermore National Laboratory（劳伦斯利弗莫尔国家实验室）；College of Health Solutions, Arizona State University（亚利桑那州立大学健康解决方案学院）；School of Electrical, Computer and Energy Engineering, Arizona State University（亚利桑那州立大学电气、计算机与能源工程学院）
Smooth-Distill: A Self-distillation Framework for Multitask Learning with Wearable Sensor Data
Faculty of EEE, Phenikaa School of Engineering（电子工程系，Phenikaa 工程学院）；Phenikaa University（Phenikaa 大学）；Graduate University of Science and Technology, VAST（科技研究生大学，VAST）；Institute of Information Technology（信息技术研究所）；College of Engineering & Computer Science and VinUni-Illinois Smart Health Center（工程与计算机科学学院及VinUni-伊利诺伊智能健康中心）；AGEIS, Université Grenoble Alpes（AGEIS，格勒诺布尔阿尔卑斯大学）；Institut Universitaire de France（法国大学研究院）
Ground Reaction Force Estimation via Time-aware Knowledge Distillation
Department of Computer Science and Engineering, Seoul National University of Science and Technology（首尔科学技术大学计算机科学与工程系）；Geometric Media Lab, School of Arts, Media and Engineering and School of Electrical, Computer and Energy Engineering, Arizona State University（亚利桑那州立大学几何媒体实验室、艺术、媒体与工程学院和电气、计算机与能源工程学院）；School for Engineering of Matter, Transport and Energy, Arizona State University（亚利桑那州立大学工程科学学院）；Department of Computer Science and Engineering, University of Nevada, Reno（内华达大学里诺分校计算机科学与工程系）
Selective Correlation Based Knowledge Distillation for Ground Reaction Force Estimation
Department of Computer Science and Engineering, Seoul National University of Science and Technology（首尔科学技术大学计算机科学与工程系）；Geometric Media Lab, School of Computing and Augmented Intelligence, Arizona State University（亚利桑那州立大学几何媒体实验室）；School for Engineering of Matter, Transport and Energy, Arizona State University（亚利桑那州立大学工程与能量科学学院）
Learning from Stochastic Teacher Representations Using Student-Guided Knowledge Distillation
LIVIA, Dept. of Systems Engineering, ETS Montreal（LIVIA，系统工程系，蒙特利尔工程学院）；CentraleSupélec, Universite Paris Saclay（中央理工巴黎高等学院）；Aiim Lab, Queen’s University（人工智能实验室，皇后大学）
Approaching Low-Cost Cardiac Intelligence with Semi-Supervised Knowledge Distillation
Department of Biomedical Engineering（生物医学工程系）；City University of Hong Kong（香港城市大学）；Hong Kong Center for Cerebro-Cardiovascular Health Engineering（香港脑心血管健康工程中心）；Hong Kong Institutes of Medical Engineering（香港医学工程研究所）；Chinese University of Hong Kong（香港中文大学）；McMaster University（麦 master 大学）
L2COcc: Lightweight Camera-Centric Semantic Scene Completion via Distillation of LiDAR Model
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学网络系统与控制研究所）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）；Zhejiang Guoli Security Technology Co., Ltd.（浙江国利安全科技有限公司）
National Yang Ming Chiao Tung University（阳明交通大学）；Academia Sinica（中央研究院）；Fu Jen Catholic University（辅仁大学）
Fall accidents are critical issues in an aging and aged society. Recently, many researchers developed pre-impact fall detection systems using deep learning to support wearable-based fall protection systems for preventing severe injuries. However, most works only employed simple neural network models instead of complex models considering the usability in resource-constrained mobile devices and strict latency requirements. In this work, we propose a novel pre-impact fall detection via CNN-ViT knowledge distillation, namely PreFallKD, to strike a balance between detection performance and computat…
Deep palmprint recognition has become an emerging issue with great potential for personal authentication on handheld and wearable consumer devices. Previous studies of palmprint recognition are mainly based on constrained datasets collected by dedicated devices in controlled environments, which has to reduce the flexibility and convenience. In addition, general deep palmprint recognition algorithms are often too heavy to meet the real-time requirements of embedded system. In this paper, a new palmprint benchmark is established, which consists of more than 20,000 images collected by 5 brands of…
Electroencephalogram (EEG) has been one of the common neuromonitoring modalities for real-world brain-computer interfaces (BCIs) because of its non-invasiveness, low cost, and high temporal resolution. Recently, light-weight and portable EEG wearable devices based on low-density montages have increased the convenience and usability of BCI applications. However, loss of EEG decoding performance is often inevitable due to reduced number of electrodes and coverage of scalp regions of a low-density EEG montage. To address this issue, we introduce knowledge distillation (KD), a learning mechanism d…
COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition
The University of New South Wales（新南威尔士大学）；King Mongkut's University of Technology North Bangkok（科技技术北巴吞蓬大学）；The Hong Kong University of Science（香港科学大学）
We explore the use of knowledge distillation (KD) for learning compact and accurate models that enable classification of animal behavior from accelerometry data on wearable devices. To this end, we take a deep and complex convolutional neural network, known as residual neural network (ResNet), as the teacher model. ResNet is specifically designed for multivariate time-series classification. We use ResNet to distill the knowledge of animal behavior classification datasets into soft labels, which consist of the predicted pseudo-probabilities of every class for each datapoint. We then use the sof…
Optimising TinyML with Quantization and Distillation of Transformer and Mamba Models for Indoor Localisation on Edge Devices
School of Engineering Mathematics and Technology, University of Bristol（布里斯托大学工程数学与技术学院）；School of Natural and Computing Sciences, University of Aberdeen（阿伯丁大学自然与计算科学学院）
Southern University of Science and Technology（南方科技大学）；The University of British Columbia（不列颠哥伦比亚大学）
Recognizing human locomotion intent and activities is important for controlling the wearable robots while walking in complex environments. However, human-robot interface signals are usually user-dependent, which causes that the classifier trained on source subjects performs poorly on new subjects. To address this issue, this paper designs the ensemble diverse hypotheses and knowledge distillation (EDHKD) method to realize unsupervised cross-subject adaptation. EDH mitigates the divergence between labeled data of source subjects and unlabeled data of target subjects to accurately classify the l…
Human Activity Recognition~(HAR) is the classification of human movement, captured using one or more sensors either as wearables or embedded in the environment~(e.g. depth cameras, pressure mats). State-of-the-art methods of HAR rely on having access to a considerable amount of labelled data to train deep architectures with many train-able parameters. This becomes prohibitive when tasked with creating models that are sensitive to personal nuances in human movement, explicitly present when performing exercises. In addition, it is not possible to collect training data to cover all possible subje…
