发表机构
Dalian University of Technology; The University of Tokyo（大连理工大学; 东京大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对光学条件受限的遥操作环境，提出一种集成sEMG和IMU的可穿戴多模态人机接口MI-DHMI，通过多模态深度学习实现手势、抓取力和手部姿态的同步高精度解码，显著提升鲁棒性。
AI 中文摘要
在普遍存在且光学条件具有挑战性的遥操作环境中，一种将可穿戴性与手部意图（手部姿态、手势和抓取力）精确解码相结合、用于遥操作抓取的接口至关重要。然而，现有接口往往无法满足这些需求，要么损害多意图解码的多样性，要么损害可穿戴性。为解决这一问题，我们开发了一种新型多意图解码人机接口（MI-DHMI），该接口集成了高通量表面肌电（sEMG）传感器以及手戴式和前臂戴式惯性测量单元（IMU）。所开发的接口由一个用于同时多意图解码的统一框架支持。通过采用多模态深度学习以及具有低噪声底层的硬件设计，该解码框架选择性地关注与手指运动真正相关的sEMG成分。这有效减少了在无约束上肢运动期间由sEMG变异性引起的解码误差，从而显著增强了鲁棒性。即使在无约束的手腕和前臂运动下，该接口也实现了超过97%的手势识别准确率、$R^2 = 0.95$的抓取力估计，以及与真实手部姿态一致的手部姿态解码，优于基线设备和算法。消融研究进一步验证了所提解码框架的有效性。最后，进行了两项在线实验来验证该设备，展示了其在包括倒水任务和物体抓取在内的高稳定性任务中的优越性能。所开发的接口为全可穿戴、多意图解码系统提供了一种新解决方案，为普遍存在的遥操作提供了有效支持，并有助于推进人机交互研究。
英文摘要
Under ubiquitous teleoperation environments with optically challenging conditions, an interface for tele-operated grasping that combines wearability with precise decoding of hand intentions (hand pose, gestures, and grasping force) is essential. Yet, existing interfaces often fall short in meeting these demands, compromising either the diversity of multiple intentions decoding or wearability. To address this, we developed a novel Multiple Intentions Decoding Human-Machine Interface (MI-DHMI) that integrates high-throughput surface electromyography (sEMG) sensors with hand-mounted and forearm-mounted inertial measurement units (IMUs). The developed interface is supported by a unified framework for simultaneous multiple intentions decoding. By employing multimodal deep learning and hardware design with a low noise floor, the decoding framework selectively focuses on the sEMG components that are genuinely associated with finger movements. This effectively reduces decoding errors caused by sEMG variability during unconstrained upper-limb motions, thereby significantly enhancing robustness. Even under unconstrained wrist and forearm motion, the interface achieves a gesture recognition accuracy exceeding 97%, grasping force estimation with $R^2 = 0.95$, and hand pose decoding consistent with the actual hand pose, outperforming baseline devices and algorithms. Ablation studies further validate the effectiveness of the proposed decoding framework. Finally, two online experiments were conducted to validate the device, demonstrating its superior performance in high-stability tasks, including a pouring task and object grasping. The developed interface provides a new solution of a fully wearable, multiple intentions decoding system, offering effective support for ubiquitous teleoperation and contributing to the advancement of human-machine interaction research.
