arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.00945cs.LG
收藏

IMG2IMU:将大规模图像中的知识迁移到 IMU 传感应用

IMG2IMU: Translating Knowledge from Large-Scale Images to IMU Sensing Applications

KAIST(韩国科学技术院);VinUni-Illinois Smart Health Center(VinUni-伊利诺伊智能健康中心);Nokia Bell Labs(诺基亚贝尔实验室)

Hyungjun Yoon, Hyeongheon Cha, Hoang C. Nguyen, Taesik Gong, Sung-Ju Lee

AI总结:

针对 IMU 预训练数据不足的问题,本文提出 IMG2IMU,通过频谱图和传感器感知图像对比预训练迁移视觉知识,在四项任务上平均提升 F1 分数 9.6 个百分点。

摘要:

通过自监督学习获得的预训练表示,即使在训练数据很少的任务上也能实现较高准确率。与视觉和自然语言处理领域不同,基于 IMU 的应用进行预训练颇具挑战,因为几乎没有规模和多样性足以学习可泛化表示的公开数据集。为克服这一问题,我们提出 IMG2IMU,将从大规模图像中预训练得到的表示适配到多种 IMU 传感任务。我们将传感器数据转换为视觉上可解释的频谱图,使模型能够利用从视觉中获得的知识。我们进一步提出一种面向图像的传感器感知预训练方法,使模型获得对 IMU 传感应用特别有价值的知识;该方法包括在针对传感器数据特性定制的增强集合上使用对比学习。我们在四个不同的 IMU 传感任务上进行评估,结果表明 IMG2IMU 相比在传感器数据上预训练的基线,在 F1 分数上平均高出 9.6 个百分点,说明视觉知识可以有效融入仅有有限训练数据可用的 IMU 传感应用。

英文摘要:

Pre-training representations acquired via self-supervised learning could achieve high accuracy on even tasks with small training data. Unlike in vision and natural language processing domains, pre-training for IMU-based applications is challenging, as there are few public datasets with sufficient size and diversity to learn generalizable representations. To overcome this problem, we propose IMG2IMU that adapts pre-trained representation from large-scale images to diverse IMU sensing tasks. We convert the sensor data into visually interpretable spectrograms for the model to utilize the knowledge gained from vision. We further present a sensor-aware pre-training method for images that enables models to acquire particularly impactful knowledge for IMU sensing applications. This involves using contrastive learning on our augmentation set customized for the properties of sensor data. Our evaluation with four different IMU sensing tasks shows that IMG2IMU outperforms the baselines pre-trained on sensor data by an average of 9.6%p F1-score, illustrating that vision knowledge can be usefully incorporated into IMU sensing applications where only limited training data is available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑