从手视角视觉信息到抓握类型概率：通过排名标签的深度学习
From Hand-Perspective Visual Information to Grasp Type Probabilities: Deep Learning via Ranking Labels
摘要：
肢体缺陷严重影响了截肢者的日常生活，并促使人们努力提供功能性机器人假手来弥补这一缺陷。基于卷积神经网络的计算机视觉假手控制由于其通过训练视觉信息来预测手势的可靠性而作为替代或补充生理信号的方法受到越来越多的关注。将摄像机安装在假手的手掌上被证明是一种很有前途的视觉数据采集方法。然而，从眼睛和手的角度标记的抓取类型可能不同，因为对象形状并不总是对称的。因此，为了以现实的方式表示这种差异，我们采用了包含来自眼睛和手视图的同步图像的数据集，其中手视角图像用于训练，而眼睛视图图像仅用于手动标记。此外，还收集了上臂的肌电图（EMG）活动和运动学数据，以便在未来的工作中进行多模态信息融合。此外，为了包括人在回路控制，并结合计算机视觉与生理信号输入，而不是作出绝对的积极或消极的预测，我们建立了一个新的概率分类器根据Plackett-Luce模型。为了预测把握的概率分布，我们利用标签排名的统计模型，通过最大似然估计来解决排列域问题，利用手动排名的把握列表作为一种新形式的标签。我们表明，该模型适用于最流行和生产力的卷积神经网络框架。
英文摘要：
Limb deficiency severely affects the daily lives of amputees and drives efforts to provide functional robotic prosthetic hands to compensate this deprivation. Convolutional neural network-based computer vision control of the prosthetic hand has received increased attention as a method to replace or complement physiological signals due to its reliability by training visual information to predict the hand gesture. Mounting a camera into the palm of a prosthetic hand is proved to be a promising approach to collect visual data. However, the grasp type labelled from the eye and hand perspective may differ as object shapes are not always symmetric. Thus, to represent this difference in a realistic way, we employed a dataset containing synchronous images from eye- and hand- view, where the hand-perspective images are used for training while the eye-view images are only for manual labelling. Electromyogram (EMG) activity and movement kinematics data from the upper arm are also collected for multi-modal information fusion in future work. Moreover, in order to include human-in-the-loop control and combine the computer vision with physiological signal inputs, instead of making absolute positive or negative predictions, we build a novel probabilistic classifier according to the Plackett-Luce model. To predict the probability distribution over grasps, we exploit the statistical model over label rankings to solve the permutation domain problems via a maximum likelihood estimation, utilizing the manually ranked lists of grasps as a new form of label. We indicate that the proposed model is applicable to the most popular and productive convolutional neural network frameworks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
