DexSim2Real$^{2}$：构建显式世界模型以实现精确的铰接物体灵巧操作
DexSim2Real$^{2}$: Building Explicit World Model for Precise Articulated Object Dexterous Manipulation
AI总结：
提出DexSim2Real$^{2}$框架，通过主动交互构建未见铰接物体的显式世界模型，结合基于采样的模型预测控制与3D AIGC数字孪生，在无需演示或强化学习的情况下实现了多种夹爪与灵巧手的精确操作及工具使用。
摘要：
铰接物体在日常生活中无处不在。本文提出了DexSim2Real$^{2}$，一种用于目标条件化铰接物体操作的新颖框架。该框架的核心是通过主动交互构建未见铰接物体的显式世界模型，从而支持基于采样的模型预测控制，在无需演示或RL的情况下规划实现不同目标的轨迹。它首先利用在自监督交互数据或人类操作视频上训练的affordance network来预测交互。在真实机器人上执行交互以移动物体部件后，我们提出了一种基于3D AIGC的新颖建模流程，从多帧观测中在仿真中构建物体的数字孪生。对于灵巧手，我们利用eigengrasp来降低动作维度，从而实现更高效的轨迹搜索。实验验证了该框架在使用吸力夹爪、两指夹爪和两种灵巧手进行精确操作时的有效性。显式世界模型的泛化能力还支持诸如使用工具操作等高级操作策略。
英文摘要：
Articulated objects are ubiquitous in daily life. In this paper, we present DexSim2Real$^{2}$, a novel framework for goal-conditioned articulated object manipulation. The core of our framework is constructing an explicit world model of unseen articulated objects through active interactions, which enables sampling-based model predictive control to plan trajectories achieving different goals without requiring demonstrations or RL. It first predicts an interaction using an affordance network trained on self-supervised interaction data or videos of human manipulation. After executing the interactions on the real robot to move the object parts, we propose a novel modeling pipeline based on 3D AIGC to build a digital twin of the object in simulation from multiple frames of observations. For dexterous hands, we utilize eigengrasp to reduce the action dimension, enabling more efficient trajectory searching. Experiments validate the framework's effectiveness for precise manipulation using a suction gripper, a two-finger gripper and two dexterous hand. The generalizability of the explicit world model also enables advanced manipulation strategies like manipulating with tools.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
