PartInstruct：面向细粒度机器人操作的部件级指令跟随
PartInstruct: Part-level Instruction Following for Fine-grained Robot Manipulation
AI总结：
本研究推出首个面向细粒度机器人操作的部件级指令基准PartInstruct，包含丰富标注的物体实例与任务、合成专家演示及测试套件，实验发现现有模型在部件概念落地和长时序部件操作上存在不足。
摘要：
细粒度机器人操作，例如拿起并旋转瓶子以展示瓶盖上的标签，需要对物体部件及其与目标任务的关系进行鲁棒推理。尽管近期在以语言指令为引导训练通用机器人操作策略方面取得了进展，但目前仍明显缺乏带有部件级指令、且标注了部件级标签的多样化3D物体实例的大规模细粒度操作任务数据集。在本研究中，我们推出了PartInstruct，这是首个用于训练和评估采用部件级指令的细粒度机器人操作模型的大规模基准。PartInstruct包含14个类别下的513个物体实例，每个实例都标注了部件级信息，还包含1302项细粒度操作任务，这些任务被划分为16个任务类别。我们的训练集包含在3D模拟器中合成的10000余条专家演示，每条演示都配对了高层任务指令、基于基础部件的技能指令链，以及物体及其部件的真实3D信息。此外，我们设计了一套全面的测试套件，用于评估所学策略在新状态、新物体和新任务上的泛化能力。我们在该基准上评估了多种当前最优的机器人操作方法，包括端到端视觉-语言策略学习和机器人操作的双层规划模型。实验结果表明，现有模型难以鲁棒地将部件概念落地并在3D空间中预测动作，且在长时序任务中操作物体部件时面临挑战。
英文摘要：
Fine-grained robot manipulation, such as lifting and rotating a bottle to display the label on the cap, requires robust reasoning about object parts and their relationships with intended tasks. Despite recent advances in training general-purpose robot manipulation policies guided by language instructions, there is a notable lack of large-scale datasets for fine-grained manipulation tasks with part-level instructions and diverse 3D object instances annotated with part-level labels. In this work, we introduce PartInstruct, the first large-scale benchmark for training and evaluating fine-grained robot manipulation models using part-level instructions. PartInstruct comprises 513 object instances across 14 categories, each annotated with part-level information, and 1302 fine-grained manipulation tasks organized into 16 task classes. Our training set consists of over 10,000 expert demonstrations synthesized in a 3D simulator, where each demonstration is paired with a high-level task instruction, a chain of base part-based skill instructions, and ground-truth 3D information about the object and its parts. Additionally, we designed a comprehensive test suite to evaluate the generalizability of learned policies across new states, objects, and tasks. We evaluated several state-of-the-art robot manipulation approaches, including end-to-end vision-language policy learning and bi-level planning models for robot manipulation on our benchmark. The experimental results reveal that current models struggle to robustly ground part concepts and predict actions in 3D space, and face challenges when manipulating object parts in long-horizon tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
