EgoEMG：一个用于手姿态估计的多模态自体视角数据集，包含双侧EMG和视觉信息
EgoEMG: A Multimodal Egocentric Dataset with Bilateral EMG and Vision for Hand Pose Estimation
AI总结：
EgoEMG数据集融合双侧EMG和视觉信息，用于手姿态估计，包含16通道EMG、IMU、RGB视频和RGB-D视频，涵盖41名参与者60种手势，提供EMG-to-pose、vision-to-pose和融合任务的基准测试。
摘要：
表面肌电图（sEMG）记录手部运动时的肌肉活动，并可通过解码恢复详细的手部姿态。EMG和自体视觉在手部感知中互补：EMG在遮挡和光照差的情况下仍能捕捉精细的手指姿态，而视觉提供全局手部配置。然而，现有的数据集未能同步这两种模态。我们提出了EgoEMG，一个多模态自体视角数据集，用于双手姿态估计。EgoEMG包括双侧腕带EMG（共16个通道，每侧8个通道），采样频率为2 kHz，120 Hz的IMU，自体视角广角RGB视频，外部RGB-D视频，以及运动捕捉导出的手部运动和腕关节角度。该数据集覆盖41名参与者执行的60种手势类别，包括30种单手手势和30种双手手势，总记录时间超过10小时。我们还引入了一个基准，包含三个任务——EMG-to-pose、vision-to-pose和EMG+vision融合——在共享的关节角预测目标和共同的泛化分割轴（跨手势、跨用户和结合）下。作为基线，我们评估了EMGFormer用于EMG-to-pose和通用的ResNet/ViT后端用于vision-to-pose。我们进一步研究了一种残差融合架构，该架构在匹配的轻量视觉-only基线之上有所改进。共同，EgoEMG及其基准为未来研究多模态手姿态估计提供了基础。
英文摘要：
Surface electromyography (sEMG) records muscle activity during hand movement and can be decoded to recover detailed hand articulation. EMG and egocentric vision are complementary for hand sensing: EMG captures fine-grained finger articulation even under occlusion and poor lighting, while vision provides global hand configuration. However, no existing dataset synchronizes both modalities. We present EgoEMG, a multimodal egocentric dataset for bimanual hand pose estimation. EgoEMG includes bilateral wristband EMG with 16 total channels (8 per wrist) sampled at 2 kHz, 120 Hz IMU, egocentric wide-angle RGB video, external RGB-D video, and mocap-derived hand motion with wrist articulation angles. The dataset covers 41 participants performing 60 gesture classes, including 30 single-hand gestures and 30 bimanual gestures, totaling more than 10 hours of recording. We also introduce a benchmark with three tasks -- EMG-to-pose, vision-to-pose, and EMG+vision fusion -- under a shared joint-angle prediction target and common generalization split axes (cross-gesture, cross-user, and combined). As baselines, we evaluate EMGFormer for EMG-to-pose and generic ResNet/ViT backbones for vision-to-pose. We further study a residual fusion architecture that improves over matched lightweight vision-only baselines. Together, EgoEMG and its benchmark establish a foundation for future research on multimodal hand pose estimation with EMG and vision.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
