跨手的潜在表示用于视觉-语言-动作模型
Cross-Hand Latent Representation for Vision-Language-Action Models
浏览论文内容
中文总结 AI 辅助
XL-VLA通过统一的潜在动作空间实现跨手灵巧操作的可扩展学习，提升视觉-语言-动作模型的训练效率和性能。
中文摘要 AI 辅助
灵巧操作对于现实世界中机器人的自主性至关重要，这与人类在日常活动中手部协调的核心作用相呼应。人类依赖丰富的多模态感知——视觉、声音和语言引导的意图——来执行灵巧动作，这促使了为机器人开发基于视觉、语言条件的操控系统。然而，训练可靠的视觉-语言-动作（VLA）模型以实现灵巧操作需要在许多机器人手部上进行大规模演示。此外，随着新的灵巧 embodiment 快速出现，为每个收集数据变得成本高昂且不切实际，这产生了对可扩展的跨 embodiment 学习的需求。我们引入了XL-VLA，一种集成了统一潜在动作空间的视觉-语言-动作框架，该空间在各种灵巧手部之间共享。这种 embodiment 不变的潜在空间可以直接插入到标准VLA架构中，使无缝的跨 embodiment 训练和高效重用现有和新收集的数据成为可能。实验结果表明，XL-VLA在原始关节空间中运行的基线VLA模型上表现一致，证明了其作为可扩展跨 embodiment 灵巧操作的有效解决方案。
英文摘要
Dexterous manipulation is essential for real-world robot autonomy, mirroring the central role of human hand coordination in daily activity. Humans rely on rich multimodal perception--vision, sound, and language-guided intent--to perform dexterous actions, motivating vision-based, language-conditioned manipulation systems for robots. However, training reliable vision-language-action (VLA) models for dexterous manipulation requires large-scale demonstrations across many robotic hands. In addition, as new dexterous embodiments appear rapidly, collecting data for each becomes costly and impractical, creating a need for scalable cross-embodiment learning. We introduce XL-VLA, a vision-language-action framework integrated with a unified latent action space shared across diverse dexterous hands. This embodiment-invariant latent space is directly pluggable into standard VLA architectures, enabling seamless cross-embodiment training and efficient reuse of both existing and newly collected data. Experimental results demonstrate that XL-VLA consistently outperforms baseline VLA models operating in raw joint spaces, establishing it as an effective solution for scalable cross-embodiment dexterous manipulation.
