Pre-training representations acquired via self-supervised learning could achieve high accuracy on even tasks with small training data. Unlike in vision and natural language processing domains, pre-training for IMU-based applications is challenging, as there are few public datasets with sufficient size and diversity to learn generalizable representations. To overcome this problem, we propose IMG2IMU that adapts pre-trained representation from large-scale images to diverse IMU sensing tasks. We convert the sensor data into visually interpretable spectrograms for the model to utilize the knowledge gained from vision. We further present a sensor-aware pre-training method for images that enables models to acquire particularly impactful knowledge for IMU sensing applications. This involves using contrastive learning on our augmentation set customized for the properties of sensor data. Our evaluation with four different IMU sensing tasks shows that IMG2IMU outperforms the baselines pre-trained on sensor data by an average of 9.6%p F1-score, illustrating that vision knowledge can be usefully incorporated into IMU sensing applications where only limited training data is available.
Overcoming the Intrinsic Performance Limitations of MEMS IMU via Diffusion-Based Generative Learning
School of Geodesy and Geomatics, Wuhan University(武汉大学测绘学院);Hubei Luojia Laboratory, Wuhan University(湖北珞珈实验室);Chinese Antarctic Center of Surveying and Mapping, Wuhan University(中国极地测绘南极科考中心)
Robust Multimodal Learning Framework For Intake Gesture Detection Using Contactless Radar and Wearable IMU Sensors
e-Media Research Lab(e-Media研究实验室);ESAT-STADIUS Division, KU Leuven(ESAT-STADIUS部门,KU莱顿大学);M-Group, DistriNet, Department of Computer Science, KU Leuven(M组、DistriNet、计算机科学系,KU莱顿大学)
Towards Generalizable Human Activity Recognition: A Survey
Hong Kong University of Science \& Technology (Guangzhou) China;SMART, Massachusetts Institute of Technology Singapore;University of New South Wales (UNSW) Australia;Hong Kong University of Science \& Technology (Guangzhou);SMART, Massachusetts Institute of Technology;University of New South Wales (UNSW)
Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals
Halıcıoğlu Data Science Institute, University of California San Diego(哈利奇奥格鲁数据科学研究所,加州大学圣地亚哥分校);Schools of Engineering, Arizona State University(工程学院,亚利桑那州立大学)
Worker Activity Recognition in Manufacturing Line Using Near-body Electric Field
German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);RPTU Kaiserslautern-Landau(凯泽斯劳滕-兰道莱茵兰-普法尔茨工业大学);Jožef Stefan International Postgraduate School(约热夫·斯特凡国际研究生院)