HumanVLA：迈向由视觉-语言引导的物理人形机器人物体重新排列
HumanVLA: Towards Vision-Language Directed Object Rearrangement by Physical Humanoid
AI总结：
提出HumanVLA框架，通过师生架构结合强化学习与行为克隆，实现视觉和语言引导的物理人形机器人通用物体重新排列，并引入新数据集验证其有效性。
摘要：
物理人-场景交互（HSI）在众多应用中发挥着关键作用。然而，现有的HSI技术受限于特定物体动力学和特权信息，阻碍了更全面应用的发展。为解决这一局限，我们提出了HumanVLA，用于在视觉和语言引导下进行通用物体重新排列。我们采用师生框架来开发HumanVLA。首先利用目标条件强化学习和对抗运动先验训练基于状态的教师策略。然后，通过行为克隆将其蒸馏为视觉-语言-动作模型。我们提出了几项关键见解以促进大规模学习过程。为支持物理人形机器人进行通用物体重新排列，我们引入了包含各种重新排列任务的全新Human-in-the-Room数据集。通过大量实验与分析，我们证明了所提方法的有效性。
英文摘要：
Physical Human-Scene Interaction (HSI) plays a crucial role in numerous applications. However, existing HSI techniques are limited to specific object dynamics and privileged information, which prevents the development of more comprehensive applications. To address this limitation, we introduce HumanVLA for general object rearrangement directed by practical vision and language. A teacher-student framework is utilized to develop HumanVLA. A state-based teacher policy is trained first using goal-conditioned reinforcement learning and adversarial motion prior. Then, it is distilled into a vision-language-action model via behavior cloning. We propose several key insights to facilitate the large-scale learning process. To support general object rearrangement by physical humanoid, we introduce a novel Human-in-the-Room dataset encompassing various rearrangement tasks. Through extensive experiments and analysis, we demonstrate the effectiveness of the proposed approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
