arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2507.00833cs.ROcs.AI

HumanoidGen:基于大语言模型推理的双臂灵巧操作数据生成

HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoning

Fudan University(复旦大学);Institute of Artificial Intelligence (TeleAI), China Telecom(人工智能研究所(TeleAI),中国电信);University of Science and Technology of China(中国科学技术大学);East China University of Science and Technology(东华大学)

Zhi Jing, Siyuan Yang, Jicong Ao, Ting Xiao, Yu-Gang Jiang, Chenjia Bai

AI总结:

针对人形机器人双臂灵巧操作缺乏高质量仿真任务与演示数据的问题,提出HumanoidGen框架,结合原子操作、LLM推理与改进蒙特卡洛树搜索自动生成任务与演示,实验验证生成数据可提升扩散策略性能。

摘要:

在机器人操作领域,现有机器人数据集与仿真基准主要面向机械臂平台。但对于配备双臂与灵巧手的人形机器人而言,仿真任务与高质量演示数据存在明显缺口。双臂灵巧操作本身复杂度更高,需要手臂运动与手部操作协同配合,导致自主数据采集难度较大。本文提出HumanoidGen自动化任务创建与演示采集框架,该框架利用原子灵巧操作与大语言模型(LLM)推理生成关联约束。具体而言,我们基于原子操作为所有物体资产与灵巧手添加空间标注,并调用LLM规划器,根据物体可供性与场景生成可执行的手臂运动空间约束链。为进一步提升规划能力,我们采用蒙特卡洛树搜索的变体,针对长时序任务与标注不足场景增强LLM推理效果。实验中,我们构建了包含增强场景的全新基准以评估采集数据的质量,结果显示2D与3D扩散策略的性能可随生成数据集规模提升而扩展。项目主页为https://openhumanoidgen.github.io。

英文摘要:

For robotic manipulation, existing robotics datasets and simulation benchmarks predominantly cater to robot-arm platforms. However, for humanoid robots equipped with dual arms and dexterous hands, simulation tasks and high-quality demonstrations are notably lacking. Bimanual dexterous manipulation is inherently more complex, as it requires coordinated arm movements and hand operations, making autonomous data collection challenging. This paper presents HumanoidGen, an automated task creation and demonstration collection framework that leverages atomic dexterous operations and LLM reasoning to generate relational constraints. Specifically, we provide spatial annotations for both assets and dexterous hands based on the atomic operations, and perform an LLM planner to generate a chain of actionable spatial constraints for arm movements based on object affordances and scenes. To further improve planning ability, we employ a variant of Monte Carlo tree search to enhance LLM reasoning for long-horizon tasks and insufficient annotation. In experiments, we create a novel benchmark with augmented scenarios to evaluate the quality of the collected data. The results show that the performance of the 2D and 3D diffusion policies can scale with the generated dataset. Project page is https://openhumanoidgen.github.io.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑