基于视觉的手影法用于机器人操作的逆运动学
Vision-Based Hand Shadowing for Robotic Manipulation via Inverse Kinematics
AI总结：
本文提出了一种基于单目RGB-D相机的逆运动学映射方法，用于将人类手部动作转化为机器人关节指令，通过实验验证了其在结构化和非结构化环境中的有效性。
摘要：
低成本机器人操作的远程控制仍具有挑战性，因为将人类手部运动映射到机器人关节命令困难。我们提出了一种离线手影逆运动学（IK）映射流水线，该流水线由一个安装在3D打印眼镜上的单目RGB-D相机驱动。该流水线使用MediaPipe Hands检测每只手的21个手部地标，通过深度感应将其投影到3D空间，转换为机器人坐标系，并解决一个阻尼最小二乘IK问题，以产生SO-ARM101机器人（5个臂+1个夹具关节）的关节指令。夹具控制器将拇指-食指几何形状映射到抓取开口，通过多级回退层次实现。动作在物理模拟中预览后再在物理机器人上播放。我们评估了该流水线在结构化拾取和放置基准（5块瓷砖网格，每块瓷砖10次抓取，3次独立运行）上，实现了86.7%±4.2%的成功率，并与四个基于视觉-语言-动作（VLA）策略（ACT、SmolVLA、pi_0.5、GR00T N1.5）进行比较，这些策略是在领导者-追随者远程操作数据上训练的。我们提供了流水线的定量误差分析，报告了平均IK位置误差为36.4毫米，轨迹平滑度指标显示从EMA平滑处理中减少了57-68%的冲击，以及对平滑参数的消融研究。我们还测试了该流水线在非结构化现实环境中（杂货店、药房）的效果，并发现由于周围物体遮挡手部，成功率降至9.3%。为缓解这一问题，我们集成了WiLoR作为替代的手部检测器，实现了比MediaPipe高8%的手部检测率，突显了无标记分析映射的潜力和当前限制。
英文摘要：
Teleoperation of low-cost robotic manipulators remains challenging due to the difficulty of retargeting human hand motion to robot joint commands. We present an offline hand-shadowing inverse-kinematics (IK) retargeting pipeline driven by a single egocentric RGB-D camera mounted on 3D-printed glasses. The pipeline detects 21 hand landmarks per hand using MediaPipe Hands, deprojects them into 3D via depth sensing, transforms them into the robot coordinate frame, and solves a damped-least-squares IK problem to produce joint commands for the SO-ARM101 robot (5 arm + 1 gripper joints). A gripper controller maps thumb-index finger geometry to grasp aperture with a multi-level fallback hierarchy. Actions are previewed in a physics simulation before replay on the physical robot. We evaluate the pipeline on a structured pick-and-place benchmark (5-tile grid, 10 grasps per tile, 3 independent runs) achieving an 86.7% +/- 4.2% success rate, and compare it against four vision-language-action (VLA) policies (ACT, SmolVLA, pi_0.5, GR00T N1.5) trained on leader-follower teleoperation data. We provide a quantitative error analysis of the pipeline, reporting a mean IK position error of 36.4 mm, trajectory smoothness metrics showing 57-68% jerk reduction from EMA smoothing, and an ablation study over the smoothing parameter. We also test the pipeline in unstructured real-world environments (grocery store, pharmacy) and find that success is reduced to 9.3% due to hand occlusion by surrounding objects. To mitigate this, we integrate WiLoR as an alternative hand detector, achieving an 8% improvement in hand detection rate over MediaPipe, highlighting both the promise and current limitations of marker-free analytical retargeting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
