COMODO：跨模态视频到IMU知识蒸馏用于高效的第一人称人类活动识别
COMODO: Cross-Modal Video-to-IMU Distillation for Efficient Egocentric Human Activity Recognition
AI总结：
本文提出COMODO框架，通过跨模态自监督蒸馏将视频语义知识转移到IMU，提升第一人称人类活动识别的效率与泛化能力。
摘要：
创建智能、以人为本的可穿戴系统以实现持续活动理解面临根本性的权衡：基于第一人称视频的模型能捕捉丰富的语义信息并在人类活动识别（HAR）中表现出色，但其高能耗、隐私问题和对光照的依赖限制了持续设备端识别的可行性。相比之下，惯性测量单元（IMU）传感器提供了一种节能、隐私保护的替代方案，但缺乏大规模标注数据集，导致泛化能力较弱。为弥合这一差距，我们提出了COMODO，一种跨模态自监督蒸馏框架，该框架无需标签即可将语义知识从视频转移到IMU。COMODO利用预训练并冻结的视频编码器构建动态实例队列，对视频和IMU嵌入的特征分布进行对齐。这使IMU编码器能够继承丰富的语义结构，同时保持其在现实应用中的效率。在多个第一人称HAR数据集上的实验表明，COMODO在下游任务中表现一致，与全监督模型匹配或超越，并显示出强大的跨数据集泛化能力。得益于其简洁性和灵活性，COMODO兼容多种预训练视频和时间序列模型，为未来无处不在的计算研究提供了潜在的利用更强大教师和学生基础模型的可能性。代码可在本仓库中获得：https://github.com/cruiseresearchgroup/COMODO。
英文摘要：
The goal of creating intelligent, human-centered wearable systems for continuous activity understanding faces a fundamental trade-off: Egocentric video-based models capture rich semantic information and have demonstrated strong performance in human activity recognition (HAR), but their high power consumption, privacy concerns, and dependence on lighting limit their feasibility for continuous on-device recognition. In contrast, inertial measurement unit (IMU) sensors offer an energy-efficient, privacy-preserving alternative, yet lack large-scale annotated datasets, leading to weaker generalization. To bridge this gap, we propose COMODO, a cross-modal self-supervised distillation framework that transfers semantic knowledge from video to IMU without requiring labels. COMODO leverages a pretrained and frozen video encoder to construct a dynamic instance queue to align the feature distributions of video and IMU embeddings. This enables the IMU encoder to inherit rich semantic structure from video while maintaining its efficiency for real-world applications. Experiments on multiple egocentric HAR datasets show that COMODO consistently improves downstream performance, matching or surpassing fully supervised models, and demonstrating strong cross-dataset generalization. Benefiting from its simplicity and flexibility, COMODO is compatible with diverse pretrained video and time-series models, offering the potential to leverage more powerful teacher and student foundation models in future ubiquitous computing research. The code is available at this repository: https://github.com/cruiseresearchgroup/COMODO.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
