arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2211.15871cs.ROcs.CVcs.HC
收藏

利用前臂超声同时估计手掌和手指关节角度

Simultaneous Estimation of Hand Configurations and Finger Joint Angles using Forearm Ultrasound

Nokia Bell Labs(诺基亚贝尔实验室);Worcester Polytechnic Institute(伍斯特理工学院);Amazon Robotics(亚马逊机器人)

Keshav Bimbraw, Christopher J. Nycz, Matt Schueler, Ziming Zhang, Haichong K. Zhang

摘要:

随着计算和机器人技术的进步,有必要开发流畅和直观的方法来与数字系统、增强/虚拟现实(AR/VR)界面和物理机器人系统进行交互。手部动作识别被广泛用于实现这些交互。手部形态分类和MCP关节角度检测是全面重建手部运动的重要环节。sEMG和其他技术已经用于检测手部运动。前臂超声图像提供了一个肌肉骨骼可视化,可用于了解手的运动。最近的工作表明,这些超声图像可以使用机器学习来分类,以估计离散的手配置。基于前臂超声估计手部构型和MCP关节角度在文献中尚未提及。在本文中,我们提出了一个基于CNN的深度学习管道来预测MCP关节角度。通过使用不同的机器学习算法的手的配置分类的结果进行了比较。SVC与不同的内核,MLP,和建议的CNN已被用来分类的超声图像到11手配置的基础上的日常生活活动。前臂超声图像采集从6名受试者指示移动他们的手根据预定义的手配置。采集运动捕捉数据,得到不同速度下手部运动对应的手指角度。在数据集的一个子集上观察到所提出的CNN的平均分类准确率为82.7%,不同内核的SVC的平均分类准确率超过80%。在预测的和真实的MCP关节角度之间获得了7.35度的平均RMSE。已经提出了一种低延迟(6.25 - 9.1 Hz)流水线,用于估计MCP关节角度和手配置,旨在实时控制人机界面。

英文摘要:

With the advancement in computing and robotics, it is necessary to develop fluent and intuitive methods for interacting with digital systems, augmented/virtual reality (AR/VR) interfaces, and physical robotic systems. Hand motion recognition is widely used to enable these interactions. Hand configuration classification and MCP joint angle detection is important for a comprehensive reconstruction of hand motion. sEMG and other technologies have been used for the detection of hand motions. Forearm ultrasound images provide a musculoskeletal visualization that can be used to understand hand motion. Recent work has shown that these ultrasound images can be classified using machine learning to estimate discrete hand configurations. Estimating both hand configuration and MCP joint angles based on forearm ultrasound has not been addressed in the literature. In this paper, we propose a CNN based deep learning pipeline for predicting the MCP joint angles. The results for the hand configuration classification were compared by using different machine learning algorithms. SVC with different kernels, MLP, and the proposed CNN have been used to classify the ultrasound images into 11 hand configurations based on activities of daily living. Forearm ultrasound images were acquired from 6 subjects instructed to move their hands according to predefined hand configurations. Motion capture data was acquired to get the finger angles corresponding to the hand movements at different speeds. Average classification accuracy of 82.7% for the proposed CNN and over 80% for SVC for different kernels was observed on a subset of the dataset. An average RMSE of 7.35 degrees was obtained between the predicted and the true MCP joint angles. A low latency (6.25 - 9.1 Hz) pipeline has been proposed for estimating both MCP joint angles and hand configuration aimed at real-time control of human-machine interfaces.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑