从手语生成到仿人执行：具有碰撞缓解功能的视觉语言引导重定向

中文摘要 AI 辅助

近期的手语生成（SLG）系统越来越多地输出密集的3D身体表示，能更好地为人形机器人的下游实施保留全身运动学和几何形状。然而，生成的动作经常出现自相交，如手与手、手与躯干的穿透。这些在离线渲染中可容忍的瑕疵，在人形执行中很关键，会导致逆运动学（IK）解决方案不可行、碰撞和重定向轨迹不稳定。我们提出一个系统级框架，通过两个组件将SLG输出与人形关节空间执行连接起来。首先，引入一个体积SMPL-X碰撞缓解模块，将生成的手语动作投影到物理上合理的配置，同时最小化偏离原始轨迹。其次，提出一种基于IK主干的视觉语言引导重定向算法：一个VLM作为渲染人形运动的视觉评论家，识别特定实施的失败模式，并触发有针对性的任务空间校正。我们的结果突出了碰撞处理和感知引导优化是可靠人形手语中关键的缺失组件。

英文摘要

Recent sign language generation (SLG) systems increasingly output dense 3D body representations, which better preserve full-body kinematics and geometry for downstream embodiment on humanoid robots. However, these generated motions frequently exhibit self-intersections such as hand-hand and hand-torso penetration. While such artifacts may be tolerated in offline rendering, they become critical in humanoid execution as they lead to infeasible inverse-kinematics (IK) solutions, collisions, and unstable retargeted trajectories. We present a system-level framework that bridges SLG outputs to humanoid joint-space execution via two components. First, we introduce a volumetric SMPL-X collision-mitigation module that projects generated signing motions toward physically plausible configurations while minimally deviating from the original trajectory. Second, we propose a vision-language-guided retargeting algorithm built on an IK backbone: a VLM serves as a visual critic over rendered humanoid motion, identifies embodiment-specific failure modes, and triggers targeted task-space corrections. Our results highlight collision handling and perception-guided refinement as key missing components for reliable humanoid signing.

发表机构

• Department of Systems and Control Engineering, Institute of Science Tokyo（东京科学大学系统与控制工程系）
• RIKEN Guardian Robot Project（理化学研究所守护机器人项目）
• ATR Hiroshi Ishiguro Laboratories（ATR石黑浩实验室）
• RIKEN Center for Biosystems Dynamics Research（理化学研究所生物系统动力学研究中心）

机构由 AI 辅助整理，请以论文原文为准。