VoxAct-B:面向双臂操作的基于体素的动作执行与稳定策略
VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulation
AI总结:
针对双臂操作动作空间维度高、现有方法样本效率低且泛化性有限的问题,提出语言条件的体素基方法VoxAct-B,借助VLMs重建体素网格以学习动作与稳定策略,在仿真和真实任务中均表现更优。
摘要:
双臂操作在众多机器人应用中至关重要。与单臂操作相比,双臂操作任务因动作空间维度更高而极具挑战性。现有研究利用大量数据与基元动作解决该问题,但可能存在样本效率低、跨任务泛化能力有限的缺陷。为此,我们提出VoxAct-B,这是一种语言条件的、基于体素的方法,利用视觉语言模型(Vision Language Models, VLMs)对场景内关键区域进行优先级排序并重建体素网格。我们将该体素网格输入双臂操作策略,以学习动作执行与稳定动作。该方法可实现更高效的体素策略学习,且能泛化至不同任务。在仿真环境中,我们验证了VoxAct-B在细粒度双臂操作任务上的表现优于强基线方法。此外,我们使用两台UR5机器人,在真实世界的Open Drawer(开抽屉)和Open Jar(开罐)任务上演示了VoxAct-B的效果。代码、数据与视频可访问https://voxact-b.github.io获取。
英文摘要:
Bimanual manipulation is critical to many robotics applications. In contrast to single-arm manipulation, bimanual manipulation tasks are challenging due to higher-dimensional action spaces. Prior works leverage large amounts of data and primitive actions to address this problem, but may suffer from sample inefficiency and limited generalization across various tasks. To this end, we propose VoxAct-B, a language-conditioned, voxel-based method that leverages Vision Language Models (VLMs) to prioritize key regions within the scene and reconstruct a voxel grid. We provide this voxel grid to our bimanual manipulation policy to learn acting and stabilizing actions. This approach enables more efficient policy learning from voxels and is generalizable to different tasks. In simulation, we show that VoxAct-B outperforms strong baselines on fine-grained bimanual manipulation tasks. Furthermore, we demonstrate VoxAct-B on real-world $\texttt{Open Drawer}$ and $\texttt{Open Jar}$ tasks using two UR5s. Code, data, and videos are available at https://voxact-b.github.io.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。