用于检测以自我为中心的抓取任务意图的视线引导3D手部运动预测
Gaze-Guided 3D Hand Motion Prediction for Detecting Intent in Egocentric Grasping Tasks
AI总结：
提出一种视线引导的3D手部运动预测方法，结合历史运动与环境数据，利用向量量化变分自编码器和自回归生成Transformer预测手部姿态以检测抓取意图，显著提升了少输入帧下的预测能力。
摘要：
手部运动预测的人类意图检测对于驱动神经康复应用中的上肢辅助机器人至关重要。然而，依赖生理信号测量的传统方法具有限制性，且通常缺乏环境上下文。我们提出了一种新方法，可预测手部姿态和关节位置的未来序列。该方法整合视线信息、历史手部运动序列和环境物体数据，无需预先了解目标抓取物体即可动态适应患者的辅助需求。具体而言，我们使用向量量化变分自编码器进行鲁棒的手部姿态编码，并采用自回归生成Transformer进行有效的手部运动序列预测。我们在健康受试者的初步研究中展示了这些新技术的可用性。为训练和评估该方法，我们收集了一个包含多名受试者对不同物体进行各类抓取动作的数据集。通过大量实验，我们证明该方法能成功预测连续的手部运动。特别是，视线信息显著增强了预测能力，尤其是在输入帧较少的情况下，突显了该方法在实际应用中的潜力。
英文摘要：
Human intention detection with hand motion prediction is critical to drive the upper-extremity assistive robots in neurorehabilitation applications. However, the traditional methods relying on physiological signal measurement are restrictive and often lack environmental context. We propose a novel approach that predicts future sequences of both hand poses and joint positions. This method integrates gaze information, historical hand motion sequences, and environmental object data, adapting dynamically to the assistive needs of the patient without prior knowledge of the intended object for grasping. Specifically, we use a vector-quantized variational autoencoder for robust hand pose encoding with an autoregressive generative transformer for effective hand motion sequence prediction. We demonstrate the usability of these novel techniques in a pilot study with healthy subjects. To train and evaluate the proposed method, we collect a dataset consisting of various types of grasp actions on different objects from multiple subjects. Through extensive experiments, we demonstrate that the proposed method can successfully predict sequential hand movement. Especially, the gaze information shows significant enhancements in prediction capabilities, particularly with fewer input frames, highlighting the potential of the proposed method for real-world applications.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
