VLK：从重建场景中的合成交互学习人形机器人全身操控
VLK: Learning Humanoid Loco-Manipulation from Synthetic Interactions in Reconstructed Scenes
AI总结：
提出VLK方法，通过3D高斯泼溅重建室内场景并合成视觉-语言-运动学数据，训练人形机器人全身操控策略，实现从仿真到真实的迁移。
摘要：
基于感知的人形机器人全身操控需要将自我中心观察和任务指令与全身运动联系起来。学习这种映射需要同步的自我中心图像、语言命令和机器人兼容的运动学轨迹，然而现有数据源无法大规模提供这种完整组合。我们通过在重建场景中合成视觉-语言-运动学（VLK）监督来解决这一瓶颈。我们的流程利用3D高斯泼溅重建公制尺度的室内环境，使用特权场景信息合成导航和物体交互轨迹，并在事后渲染配对的自我中心观察。我们无需人工干预即可生成48,000条配对轨迹，并训练一个VLK策略，该策略预测短时域的全身运动学轨迹。一个全身跟踪器将这些预测转换为物理人形机器人的动作。我们在物理Unitree G1上评估了导航和单物体运输任务，证明了重建场景中的合成交互为基于仿真到现实的感知人形机器人全身操控提供了有效的监督。项目网站：https://vision-language-kinematics.github.io/
英文摘要：
Perception-based humanoid loco-manipulation requires connecting egocentric observations and task instructions to whole-body motion. Learning this mapping requires synchronized egocentric images, language commands, and robot-compatible kinematic trajectories, yet no existing data source provides this complete tuple at scale. We address this bottleneck by generating vision-language-kinematics (VLK) supervision synthetically in reconstructed scenes. Our pipeline leverages 3D Gaussian Splatting to reconstruct metric-scale indoor environments, synthesizes navigation and object-interaction trajectories using privileged scene information, and renders paired egocentric observations after the fact. We produce 48,000 paired trajectories with no human intervention and train a VLK policy that predicts short-horizon whole-body kinematic trajectories. A whole-body tracker converts these predictions into actions on the physical humanoid. We evaluate on the physical Unitree G1 performing navigation and single-object transport, demonstrating that synthesized interactions in reconstructed scenes provide effective supervision for sim-to-real perception-based humanoid loco-manipulation. Project Website: https://vision-language-kinematics.github.io/
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
