肌电与视觉多模态融合用于假肢手控制中的人类抓取意图推断
Multimodal Fusion of EMG and Vision for Human Grasp Intent Inference in Prosthetic Hand Control
AI总结：
提出融合第一视角视频、眼动注视与前臂EMG的贝叶斯框架，利用新型数据处理和增强技术推断假肢手抓取意图，融合准确率达95.3%。
摘要：
目的：对于经桡动脉截肢者而言，机器人假肢手有望帮助其恢复执行日常生活活动的能力。当前基于肌电图（EMG）等生理信号的控制方法，由于运动伪影、肌肉疲劳等诸多因素，容易产生较差的推断结果。视觉传感器是获取环境状态信息的主要来源，可在推断可行且预期的手势方面发挥关键作用。然而，视觉证据也容易受到其自身伪影的影响，最常见原因包括物体遮挡、光照变化等。由于生理与视觉传感器测量具有互补优势，采用二者进行多模态证据融合是一种自然的方法。方法：本文提出一种贝叶斯证据融合框架，通过神经网络模型处理来自前臂的第一视角视频、眼动注视和EMG信号，以进行抓取意图推断。我们分析了当手接近物体并进行抓取时，各单一模态及融合性能随时间的变化。为此，我们还开发了用于训练神经网络组件的新型数据处理与增强技术。结果：结果表明，在伸手阶段，相较于单独的EMG（非融合准确率81.64%）和视觉证据（非融合准确率80.5%），融合平均将即时即将发生的抓取类型分类准确率分别提升13.66%和14.8%，总体融合准确率达到95.3%。结论：实验数据分析表明，EMG与视觉证据展现出互补优势，因此多模态证据融合在任意给定时刻均可优于任一单独证据模态。
英文摘要：
Objective: For transradial amputees, robotic prosthetic hands promise to regain the capability to perform daily living activities. Current control methods based on physiological signals such as electromyography (EMG) are prone to yielding poor inference outcomes due to motion artifacts, muscle fatigue, and many more. Vision sensors are a major source of information about the environment state and can play a vital role in inferring feasible and intended gestures. However, visual evidence is also susceptible to its own artifacts, most often due to object occlusion, lighting changes, etc. Multimodal evidence fusion using physiological and vision sensor measurements is a natural approach due to the complementary strengths of these modalities. Methods: In this paper, we present a Bayesian evidence fusion framework for grasp intent inference using eye-view video, eye-gaze, and EMG from the forearm processed by neural network models. We analyze individual and fused performance as a function of time as the hand approaches the object to grasp it. For this purpose, we have also developed novel data processing and augmentation techniques to train neural network components. Results: Our results indicate that, on average, fusion improves the instantaneous upcoming grasp type classification accuracy while in the reaching phase by 13.66% and 14.8%, relative to EMG (81.64% non-fused) and visual evidence (80.5% non-fused) individually, resulting in an overall fusion accuracy of 95.3%. Conclusion: Our experimental data analyses demonstrate that EMG and visual evidence show complementary strengths, and as a consequence, fusion of multimodal evidence can outperform each individual evidence modality at any given time.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
