Advancements in gesture recognition algorithms have led to a significant growth in sign language translation. By making use of efficient intelligent models, signs can be recognized with precision. The proposed work presents a novel one-dimensional Convolutional Neural Network (CNN) array architecture for recognition of signs from the Indian sign language using signals recorded from a custom designed wearable IMU device. The IMU device makes use of tri-axial accelerometer and gyroscope. The signals recorded using the IMU device are segregated on the basis of their context, such as whether they correspond to signing for a general sentence or an interrogative sentence. The array comprises of two individual CNNs, one classifying the general sentences and the other classifying the interrogative sentence. Performances of individual CNNs in the array architecture are compared to that of a conventional CNN classifying the unsegregated dataset. Peak classification accuracies of 94.20% for general sentences and 95.00% for interrogative sentences achieved with the proposed CNN array in comparison to 93.50% for conventional CNN assert the suitability of the proposed approach.
IMUs are gaining significant importance in the field of hand gesture analysis, trajectory detection and kinematic functional study. An Inertial Measurement Unit (IMU) consists of tri-axial accelerometers and gyroscopes which can together be used for formation analysis. The paper presents a novel classification approach using a Deep Neural Network (DNN) for classifying hand gestures obtained from wearable IMU sensors. An optimization objective is set for the classifier in order to reduce correlation between the activities and fit the signal-set with best performance parameters. Training of the …
Sign Language is used by the deaf community all over world. The work presented here proposes a novel one-dimensional deep capsule network (CapsNet) architecture for continuous Indian Sign Language recognition by means of signals obtained from a custom designed wearable IMU system. The performance of the proposed CapsNet architecture is assessed by altering dynamic routing between capsule layers. The proposed CapsNet yields improved accuracy values of 94% for 3 routings and 92.50% for 5 routings in comparison with the convolutional neural network (CNN) that yields an accuracy of 87.99%. Improve…
Deep Learning-Based Estimation of Ground Reaction Forces in Parkinsonian Gait Using an Optimized Set of IMU Data
University of Exeter（埃克塞特大学）；École Polytechnique Fédérale de Lausanne (EPFL)（洛桑联邦理工学院）；University of Cambridge（剑桥大学）；University of Exeter Medical School（埃克塞特大学医学院）；NIHR Exeter BRC（英国国立卫生研究院埃克塞特生物医学研究中心）
We address the well-known wearable activity recognition problem of having to work with sensors that are non-optimal in terms of information they provide but have to be used due to wearability/usability concerns (e.g. the need to work with wrist-worn IMUs because they are embedded in most smart watches). To mitigate this problem we propose a method that facilitates the use of information from sensors that are only present during the training process and are unavailable during the later use of the system. The method transfers information from the source sensors to the latent representation of th…
Inertial measurement units (IMUs) increasingly function as a basic component of wearable sensor network (WSN)systems. IMU-based joint angle estimation (JAE) is a relatively typical usage of IMUs, with extensive applications. However, the issue that IMUs move with respect to their original placement during JAE is still a research gap, and limits the robustness of deploying the technique in real-world application scenarios. In this study, we propose to detect and correct the IMU movement online in a relatively computationally lightweight manner. Particularly, we first experimentally investigate …
Robust Multimodal Learning Framework For Intake Gesture Detection Using Contactless Radar and Wearable IMU Sensors
e-Media Research Lab（e-Media研究实验室）；ESAT-STADIUS Division, KU Leuven（ESAT-STADIUS部门，KU莱顿大学）；M-Group, DistriNet, Department of Computer Science, KU Leuven（M组、DistriNet、计算机科学系，KU莱顿大学）
Wearable devices enable theoretically continuous, longitudinal monitoring of physiological measurements like step count, energy expenditure, and heart rate. Although the classification of abnormal cardiac rhythms such as atrial fibrillation from wearable devices has great potential, commercial algorithms remain proprietary and tend to focus on heart rate variability derived from green spectrum LED sensors placed on the wrist where noise remains an unsolved problem. Here, we develop a multi-task deep learning method to assess signal quality and arrhythmia event detection in wearable photoplethy…
Spatial-Related Sensors Matters: 3D Human Motion Reconstruction Assisted with Textual Semantics
Beijing Advanced Innovation Center for Materials Genome Engineering（北京材料基因组工程先进创新中心）；University of Science and Technology Beijing（北京科技大学）；Key Laboratory of Intelligent Bionic Unmanned Systems, Ministry of Education（智能仿生无人系统教育部重点实验室）；Institute of Materials Intelligent Technology, Liaoning Academy of Materials（辽宁材料实验室材料智能技术研究所）
Human Activity Recognition Based on Electrocardiogram Data Only
Department of Computer Science and Engineering, University of North Texas（计算机科学与工程系，北卡罗来纳州立大学）；Faculty of Computer Science, Technische Universität Dresden（计算机科学系，德累斯顿技术大学）
Low-power sensing technologies, such as wearables, have emerged in the healthcare domain since they enable continuous and non-invasive monitoring of physiological signals. In order to endow such devices with clinical value, classical signal processing has encountered numerous challenges. However, data-driven methods, such as machine learning, offer attractive accuracies at the expense of being resource and memory demanding. In this paper, we focus on the inference of neural networks running in microcontrollers and low-power processors which wearable sensors and devices are generally equipped w…
As market wearables are becoming popular with a range of services, including making financial transactions, accessing cars, etc. that they provide based on various private information of a user, security of this information is becoming very important. However, users are often flooded with PINs and passwords in this internet of things (IoT) world. Additionally, hard-biometric, such as facial or finger recognition, based authentications are not adaptable for market wearables due to their limited sensing and computation capabilities. Therefore, it is a time demand to develop a burden-free implici…
Technische Universität Wien (TU Wien)（维也纳工业大学）；National University of Sciences and Technology (NUST)（巴基斯坦国立科技大学）；New York University Abu Dhabi (NYU-AD)（纽约大学阿布扎比分校）
In this work, we propose the BioNetExplorer framework to systematically generate and explore multiple DNN architectures for bio-signal processing in wearables. Our framework adapts key neural architecture parameters to search for an embedded DNN with a low hardware overhead, which can be deployed in wearable edge devices to analyse the bio-signal data and to extract the relevant information, such as arrhythmia and seizure. Our framework also enables hardware-aware DNN architecture search using genetic algorithms by imposing user requirements and hardware constraints (storage, FLOPs, etc.) duri…
This paper presents a novel ECG classification algorithm for real-time cardiac monitoring on ultra low-power wearable devices. The proposed solution is based on spiking neural networks which are the third generation of neural networks. In specific, we employ spike-timing dependent plasticity (STDP), and reward-modulated STDP (R-STDP), in which the model weights are trained according to the timings of spike signals, and reward or punishment signals. Experiments show that the proposed solution is suitable for real-time operation, achieves comparable accuracy with respect to previous methods, and…
We propose cross-modal attentive connections, a new dynamic and effective technique for multimodal representation learning from wearable data. Our solution can be integrated into any stage of the pipeline, i.e., after any convolutional layer or block, to create intermediate connections between individual streams responsible for processing each modality. Additionally, our method benefits from two properties. First, it can share information uni-directionally (from one modality to the other) or bi-directionally. Second, it can be integrated into multiple stages at the same time to further allow n…
School of Electrical and Computer Engineering, University of Tehran（德黑兰大学电气与计算机工程学院）；Division of Intelligent Future Technologies, Malardalen University（马拉达伦大学智能未来技术系）
Long Short-Term Memory (LSTM) is widely used in various sequential applications. Complex LSTMs could be hardly deployed on wearable and resourced-limited devices due to the huge amount of computations and memory requirements. Binary LSTMs are introduced to cope with this problem, however, they lead to significant accuracy loss in some application such as EEG classification which is essential to be deployed in wearable devices. In this paper, we propose an efficient multi-level binarized LSTM which has significantly reduced computations whereas ensuring an accuracy pretty close to full precisio…
Wrocław University of Science and Technology（弗罗茨瓦夫理工大学）；Capgemini（凯捷）
Wearables like smartwatches or wrist bands equipped with pervasive sensors enable us to monitor our physiological signals. In this study, we address the question whether they can help us to recognize our emotions in our everyday life for ubiquitous computing. Using the systematic literature review, we identified crucial research steps and discussed the main limitations and problems in the domain.
The current dominated wearable body motion sensor is IMU. This work presented an alternative wearable motion-sensing approach: human body capacitance (HBC, also commonly defined as body-area electric field). While being less robust in tracking the posture and trajectory, HBC has two properties that make it an attractive. First, the deployment of the sensing node on the being tracked body part is not a requirement for HBC sensing approach. Second, HBC is sensitive to the body's interaction with its surroundings, including both touching and being in the immediate proximity of people and objects.…
Wearable devices are increasingly used, thanks to the wide set of applications that can be deployed exploiting their ability to monitor physical activity and health-related parameters. Their usage has been recently proposed to perform biometric recognition, leveraging on the uniqueness of the recorded traits to generate discriminative identifiers. Most of the studies conducted on this topic have considered signals derived from cardiac activity, detecting it mainly using electrical measurements thorugh electrocardiography, or optical recordings employing photoplethysmography. In this paper we i…
Objective: A novel ECG classification algorithm is proposed for continuous cardiac monitoring on wearable devices with limited processing capacity. Methods: The proposed solution employs a novel architecture consisting of wavelet transform and multiple LSTM recurrent neural networks. Results: Experimental evaluations show superior ECG classification performance compared to previous works. Measurements on different hardware platforms show the proposed algorithm meets timing requirements for continuous and real-time execution on wearable devices. Conclusion: In contrast to many compute-intensive…
