MyoSem: 将肌电图与自然语言动作语义对齐以实现手部动作理解
MyoSem: Aligning Electromyography with Natural-Language Action Semantics
AI总结：
提出MyoSem框架，通过多视角动作语义构建、激活感知EMG编码和语义查询对齐，实现EMG信号与文本描述的双向检索，在多个数据集上优于基线方法并展现良好泛化性。
摘要：
肌电图（EMG）直接反映肌肉激活，是手势识别、假肢控制和可穿戴交互的关键传感模态。然而，现有的EMG方法通常将手部动作理解视为固定标签的分类问题，难以支持基于动作描述的查询、检索和泛化。我们提出MyoSem，一个EMG-动作语义对齐框架，将低层EMG信号映射到由多视角动作描述构建的共享语义空间。MyoSem结合多视角动作语义构建、激活感知EMG编码和语义查询对齐，实现了EMG信号与文本描述之间的双向检索。我们在EMG2Pose和NinaPro系列数据集上系统评估了MyoSem。结果表明，MyoSem在EMG-文本双向检索上表现良好，普遍优于大多数基线，并在未见用户、保留动作类别和截肢用户迁移场景中展现出良好的泛化性。消融实验和可视化进一步验证了每个模块的有效性。总体而言，MyoSem将基于EMG的手部动作理解从固定标签识别推进到可查询的双向语义检索，为语言介导的EMG动作理解提供了新的建模范式。
英文摘要：
Electromyography (EMG) records muscle activation for hand gesture recognition, prosthetic control, and wearable interaction. Existing EMG methods usually formulate hand action understanding as prediction over fixed gesture labels, leaving EMG signals and action-semantic text unaligned in a common feature space and limiting bidirectional querying between signals and hand-action descriptions. We present MyoSem, an EMG--action semantic alignment framework that maps hand EMG and multi-view natural-language action descriptions into a shared feature space for retrieval. MyoSem builds semantic prototypes from multi-view descriptions, encodes EMG with local activation-aware representations, and uses semantic queries to aggregate local EMG evidence for bidirectional EMG--text retrieval. We evaluate MyoSem under seven protocols on NinaPro DB2, EMG2Pose, and NinaPro DB2$\rightarrow$DB3, covering random splits, user shifts, held-out action classes, and able-bodied-to-amputee transfer. Across this evaluation, MyoSem achieves the strongest overall bidirectional retrieval performance among the evaluated methods. Ablations and representation analyses show how semantic prototypes, activation-aware encoding, and semantic queries shape the learned space. Overall, MyoSem moves EMG hand action understanding from fixed-label recognition toward language-queryable semantic retrieval.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
