The recently-developed infant wearable MAIJU provides a means to automatically evaluate infants' motor performance in an objective and scalable manner in out-of-hospital settings. This information could be used for developmental research and to support clinical decision-making, such as detection of developmental problems and guiding of their therapeutic interventions. MAIJU-based analyses rely fully on the classification of infant's posture and movement; it is hence essential to study ways to increase the accuracy of such classifications, aiming to increase the reliability and robustness of the automated analysis. Here, we investigated how self-supervised pre-training improves performance of the classifiers used for analyzing MAIJU recordings, and we studied whether performance of the classifier models is affected by context-selective quality-screening of pre-training data to exclude periods of little infant movement or with missing sensors. Our experiments show that i) pre-training the classifier with unlabeled data leads to a robust accuracy increase of subsequent classification models, and ii) selecting context-relevant pre-training data leads to substantial further improvements in the classifier performance.
University of Southern California（南加州大学）；Universidad Carlos III de Madrid（马德里卡洛斯三世大学）
Infants with a variety of complications at or before birth are classified as being at risk for developmental delays (AR). As they grow older, they are followed by healthcare providers in an effort to discern whether they are on a typical or impaired developmental trajectory. Often, it is difficult to make an accurate determination early in infancy as infants with typical development (TD) display high variability in their developmental trajectories both in content and timing. Studies have shown that spontaneous movements have the potential to differentiate typical and atypical trajectories earl…
Aalto University（阿尔托大学）；Helsinki University Hospital（赫尔辛基大学医院）；University of Helsinki（赫尔辛基大学）；Tampere University（坦佩雷大学）；Scuola Superiore San’Anna（圣安娜高等学校）；IRCCS Fondazione Stella Maris（斯特拉·马里斯IRCCS基金会）；Kaasa solution GmbH（卡阿萨解决方案有限公司）
Infants' spontaneous and voluntary movements mirror developmental integrity of brain networks since they require coordinated activation of multiple sites in the central nervous system. Accordingly, early detection of infants with atypical motor development holds promise for recognizing those infants who are at risk for a wide range of neurodevelopmental disorders (e.g., cerebral palsy, autism spectrum disorders). Previously, novel wearable technology has shown promise for offering efficient, scalable and automated methods for movement assessment in adults. Here, we describe the development of …
Newcastle University（纽卡斯尔大学）；Great North Childrens Hospital（大北方儿童医院）；Georgia Institute of Technology（佐治亚理工学院）
Perinatal stroke (PS) is a serious condition that, if undetected and thus untreated, often leads to life-long disability, in particular Cerebral Palsy (CP). In clinical settings, Prechtl's General Movement Assessment (GMA) can be used to classify infant movements using a Gestalt approach, identifying infants at high risk of developing PS. Training and maintenance of assessment skills are essential and expensive for the correct use of GMA, yet many practitioners lack these skills, preventing larger-scale screening and leading to significant risks of missing opportunities for early detection and…
Norwegian University of Science and Technology（挪威科技大学）；St. Olavs Hospital, Trondheim University Hospital（圣奥拉夫医院，特隆赫姆大学医院）
Assessment of spontaneous movements can predict the long-term developmental disorders in high-risk infants. In order to develop algorithms for automated prediction of later disorders, highly precise localization of segments and joints by infant pose estimation is required. Four types of convolutional neural networks were trained and evaluated on a novel infant pose dataset, covering the large variation in 1 424 videos from a clinical international community. The localization performance of the networks was evaluated as the deviation between the estimated keypoint positions and human expert ann…
Infant movement classification through pressure distribution analysis
University Medical Center Göttingen（哥廷根大学医学中心）；Leibniz-ScienceCampus Primate Cognition（莱布尼茨灵长类认知科学园区）；Medical University of Graz（格拉茨医科大学）；David Geffen UCLA School of Medicine（大卫·格芬加州大学洛杉矶分校医学院）；Karolinska Institutet（卡罗林斯卡学院）；Stockholm Health Care Services（斯德哥尔摩医疗服务中心）；Curtin University（科廷大学）；Technical University Berlin（柏林工业大学）
Cerebral Palsy Alliance Research Institute（脑瘫联盟研究所）；The University of Sydney（悉尼大学）；The Children’s Hospital at Westmead（韦斯特米德儿童医院）
The absence or abnormality of fidgety movements of joints or limbs is strongly indicative of cerebral palsy in infants. Developing computer-based methods for assessing infant movements in videos is pivotal for improved cerebral palsy screening. Most existing methods use appearance-based features and are thus sensitive to strong but irrelevant signals caused by background clutter or a moving camera. Moreover, these features are computed over the whole frame, thus they measure gross whole body movements rather than specific joint/limb motion. Addressing these challenges, we develop and validate …
Northumbria University（诺森比亚大学）；NUTH NHS Foundation Trust（NUTH NHS基金会信托）
Providing early diagnosis of cerebral palsy (CP) is key to enhancing the developmental outcomes for those affected. Diagnostic tools such as the General Movements Assessment (GMA), have produced promising results in early diagnosis, however these manual methods can be laborious. In this paper, we propose a new framework for the automated classification of infant body movements, based upon the GMA, which unlike previous methods, also incorporates a visualization framework to aid with interpretability. Our proposed framework segments extracted features to detect the presence of Fidgety Movements…
Infant Spontaneous Movement Noise Improves Exploration in Deep RL
Frankfurt Institute for Advanced Studies（法兰克福高等研究所）；School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）；Escuela de Ingeniería, Universidad Central de Chile（智利中央大学工程学院）；Faculty of Electrical Engineering, Czech Technical University（捷克理工大学电气工程学院）
Deep learning empowered sensor fusion boosts infant movement classification
German Center for Child and Adolescent Health (DZKJ)（德国儿童青少年健康中心）；University Hospital Heidelberg（海德堡大学医院）；Medical University of Graz（格拉茨医科大学）；Karolinska Institutet（卡罗林斯卡学院）；Stockholm Health Care Services（斯德哥尔摩医疗服务中心）；Curtin University（科廷大学）；Technical University Berlin（柏林工业大学）；German Center for Neurodegenerative Diseases (DZNE)（德国神经退行性疾病中心）；David Geffen UCLA School of Medicine（加州大学洛杉矶分校大卫·格芬医学院）
Durham University（杜伦大学）；University of Glasgow（格拉斯哥大学）
Early prediction is clinically considered one of the essential parts of cerebral palsy (CP) treatment. We propose to implement a low-cost and interpretable classification system for supporting CP prediction based on General Movement Assessment (GMA). We design a Pytorch-based attention-informed graph convolutional network to early identify infants at risk of CP from skeletal data extracted from RGB videos. We also design a frequency-binning module for learning the CP movements in the frequency domain while filtering noise. Our system only requires consumer-grade RGB videos for training to supp…
EHESS（高等社会科学研究院）；ENS（高等师范学校）；PSL Research University（巴黎文理研究大学）；CNRS（法国国家科学研究中心）；INRIA（法国国家信息与自动化研究所）
During their first years of life, infants learn the language(s) of their environment at an amazing speed despite large cross cultural variations in amount and complexity of the available language input. Understanding this simple fact still escapes current cognitive and linguistic theories. Recently, spectacular progress in the engineering science, notably, machine learning and wearable technology, offer the promise of revolutionizing the study of cognitive development. Machine learning offers powerful learning algorithms that can achieve human-like performance on many linguistic tasks. Wearabl…
In this paper, we investigate a potential security vulnerability associated with wrist wearable devices. Hardware components on common wearable devices include an accelerometer and gyroscope, among other sensors. We demonstrate that an accelerometer and gyroscope can pick up enough unique wrist movement information to identify digits being written by a user. With a data set of 400 writing samples, of either the digit zero or the digit one, we constructed a machine learning model to correctly identify the digit being written based on the movements of the wrist. Our model's performance on an uns…
Recognizing human activities from multi-channel time series data collected from wearable sensors is ever more practical. However, in real-world conditions, coherent activities and body movements could happen at the same time, like moving head during walking or sitting. A new problem, so-called "Coherent Human Activity Recognition (Co-HAR)", is more complicated than normal multi-class classification tasks since signals of different movements are mixed and interfered with each other. On the other side, we consider such Co-HAR as a dense labelling problem that classify each sample on a time step …
Towards Scalable Newborn Screening: Automated General Movement Assessment in Uncontrolled Settings
ETH Zurich（苏黎世联邦理工学院）；University of Zurich（苏黎世大学）；University of Regensburg（莱茵河畔埃尔兰根大学）；University Children’s Hospital（儿童医院）；University Children’s Hospital Regensburg（埃尔兰根儿童医院）
California State University East Bay（加州州立大学东湾分校）；University of Alberta（阿尔伯塔大学）；San Diego State University（圣地亚哥州立大学）
Manual material handling tasks have the potential to be highly unsafe from an ergonomic viewpoint. Safety inspections to monitor body postures can help mitigate ergonomic risks of material handling. However, the real effect of awkward muscle movements, strains, and excessive forces that may result in an injury may not be identified by external cues. This paper evaluates the ability of surface electromyogram (EMG)-based systems together with machine learning algorithms to automatically detect body movements that may harm muscles in material handling. The analysis utilized a lifting equation dev…
Design of Recognition and Evaluation System for Table Tennis Players' Motor Skills Based on Artificial Intelligence
School of Electronics and Information, Northwestern Polytechnical University（西北工业大学电子信息学院）；School of Foreign Studies, Northwestern Polytechnical University（西北工业大学外国语学院）
A Foundation Model for Wearable Movement Data in Mental Health Research
Dartmouth College（达特茅斯学院）；National Institute of Diabetes and Digestive and Kidney Diseases（国家糖尿病、消化系统疾病和肾病研究所）；National Institutes of Health（美国国立卫生研究院）；Department of Computer Science at Dartmouth College（达特茅斯学院计算机科学系）
Intelligent upper-limb exoskeleton integrated with soft wearable bioelectronics and deep-learning for human intention-driven strength augmentation based on sensory feedback
Dongguk University（东国大学）；Georgia Institute of Technology（佐治亚理工学院）；Yonsei University（延世大学）；Emory University School of Medicine（埃默里大学医学院）
Optimizing Emotion Recognition with Wearable Sensor Data: Unveiling Patterns in Body Movements and Heart Rate through Random Forest Hyperparameter Tuning
Increased use of sensor signals from wearable devices as rich sources of physiological data has sparked growing interest in developing health monitoring systems to identify changes in an individual's health profile. Indeed, machine learning models for sensor signals have enabled a diverse range of healthcare related applications including early detection of abnormalities, fertility tracking, and adverse drug effect prediction. However, these models can fail to account for the dependent high-dimensional nature of the underlying sensor signals. In this paper, we introduce Latent Temporal Flows, …
Counterfactual Explanation-Based Badminton Motion Guidance Generation Using Wearable Sensors
School of Integrated Technology, Gwangjun Institute of Science and Technology（光州科学技术学院综合技术学院）；Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology（麻省理工学院计算机科学与人工智能实验室）
Assessing the health of both the fetus and mother is vital in preventing and identifying possible complications in pregnancy. This paper focuses on a device that can be used effectively by the mother herself with minimal supervision and provide a reasonable estimation of fetal and maternal health while being safe, comfortable, and easy to use. The device proposed uses a belt with a single accelerometer over the mother's uterus to record the required information. The device is expected to monitor both the mother and the fetus constantly over a long period and provide medical professionals with …
Modern wearable devices are embedded with a range of noninvasive biomarker sensors that hold promise for improving detection and treatment of disease. One such sensor is the single-lead electrocardiogram (ECG) which measures electrical signals in the heart. The benefits of the sheer volume of ECG measurements with rich longitudinal structure made possible by wearables come at the price of potentially noisier measurements compared to clinical ECGs, e.g., due to movement. In this work, we develop a statistical model to simulate a structured noise process in ECGs derived from a wearable sensor, d…
This study tackles on a new problem of estimating human-error potential on a shop floor on the basis of wearable sensors. Unlike existing studies that utilize biometric sensing technology to estimate people's internal state such as fatigue and mental stress, we attempt to estimate the human-error potential in a situation where a target person does not stay calm, which is much more difficult as sensor noise significantly increases. We propose a novel formulation, in which the human-error-potential estimation problem is reduced to a classification problem, and introduce a new method that can be …
Max Planck Institute for Informatics（马克斯·普朗克信息研究所）；Saarbrücken Research Center for Visual Computing, Interaction and AI（萨尔布吕肯视觉计算、交互与人工智能研究中心）；Google（谷歌）
VinAI Research（VinAI研究院）；Deezer Research（Deezer研究院）；Posts and Telecommunications Institute of Technology（邮政电信技术学院）
Breath with nose sound features has been shown as a potential biometric in personal identification and verification. In this paper, we show that information that comes from other modalities captured by motion sensors on the chest in addition to audio features could further improve the performance. Our work is composed of three main contributions: hardware creation, dataset publication, and proposed multimodal models. To be more specific, we design new hardware which consists of an acoustic sensor to collect audio features from the nose, as well as an accelerometer and gyroscope to collect move…
Physically Plausible Data Augmentations for Wearable IMU-based Human Activity Recognition Using Physics Simulation
School of Engineering and Informatics, University of Sussex（工程与信息学学院，Sussex大学）；Electrical and Computer Engineering Department, Concordia University（电气与计算机工程系，Concordia大学）
