GeniWorld：一种用于机器人操作的可泛化交互式世界模型
GeniWorld: A Generalizable Interactive World Model for Robotic Manipulation via Visual Actions
浏览论文内容
中文总结 AI 辅助
针对通用机器人策略在复杂环境中鲁棒性不足的问题，提出GeniWorld交互式世界模型，通过解耦本体与环境动态、构建自回归视频预测模型实现泛化，可用于策略评估并提升下游操作性能。
中文摘要 AI 辅助
通用机器人策略具备较强能力，但在复杂且未见过的环境中的鲁棒性仍有限。在多样化真实世界环境中扩展机器人学习与评估的成本高且颇具挑战性。基于动作的世界模型是一种有前景的替代方案，但它们常存在动作可控性有限、对分布外（OOD）场景泛化能力差的问题。为此，我们提出GeniWorld，一种可在未见过场景中实现稳健泛化的机器人交互式世界模型。该模型基于预训练视频生成模型构建，使用基于URDF的渲染将数值动作转换为视觉动作表示，实现空间上精准的动作控制。通过明确将机器人本体运动学与环境动态解耦，我们的模型减轻了场景过拟合，便于对机器人-环境交互进行建模。为实现闭环控制，我们构建了集成高频机器人运动学控制的自回归视频预测模型，使其可与机器人策略及人类远程操作者交互。实验中，即便仅在有限的固定场景数据上训练，我们的模型仍实现了优异的域内性能，且对高度随机的未见过环境具备稳健的零样本泛化能力。对于下游应用，GeniWorld可作为可扩展的策略评估器，在环境扰动下仍保持可靠。此外，即便仅使用有限的真实世界演示，GeniWorld也能在世界模型内生成多样化的操作轨迹，提升下游策略在复杂环境中的性能与鲁棒性。
英文摘要
Generalist robot policies exhibit strong capabilities, but their robustness in complex and unseen environments remains limited. Scaling robot learning and evaluation in diverse real-world environments remains costly and challenging. Action-conditioned world models offer a promising alternative, but they often suffer from limited action controllability and poor generalization to out-of-distribution (OOD) scenarios. To this end, we present GeniWorld, an interactive world model for robots that generalizes robustly across unseen scenarios. Building on pretrained video generative models, we use URDF-based rendering to transform numerical actions into visual action representations, enabling spatially grounded action control. By explicitly decoupling embodiment kinematics from environmental dynamics, our model mitigates scene overfitting and facilitates modeling of robot-environment interactions. To achieve closed-loop control, we construct an autoregressive video prediction model integrated with high-frequency robot kinematic control, enabling interaction with both robot policies and human teleoperators. In our experiments, even when trained solely on limited fixed-scene data, our model achieves superior in-domain performance and robust zero-shot generalization to highly randomized, unseen environments. For downstream applications, GeniWorld serves as a scalable policy evaluator that remains reliable under environmental perturbations. Furthermore, even with limited real-world demonstrations, GeniWorld generates diverse manipulation trajectories within the world model, improving downstream policy performance and robustness in complex environments.
发表机构
- Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
- Tencent Robotics X（腾讯Robotics X实验室）
- The Hong Kong University of Science and Technology（香港科技大学）
- Shenzhen Technology University（深圳技术大学）
机构由 AI 辅助整理，请以论文原文为准。
