DeformSmith：面向机器人操作的物理约束引导的可变形资产分层生成

中文摘要 AI 辅助

为机器人操作创建可变形资产需要同时指定其几何形状、外观和物理属性。这对于可变形物体尤其具有挑战性，因为文本和图像提供的关于它们如何变形以及如何响应接触的证据有限，而这些响应直接影响它们与交互的适配性。因此，自动化生成需要解决耦合的物理需求，并利用交互证据来指导构建和细化。我们提出了DeformSmith，一个能够从文本或单张图像自动生成可交互、物理可信的可变形资产的框架。通过分层智能体构建和共享的物理接地约束，它逐步构建、测试和细化几何、物理模型、材料行为以及机器人交互，直到生成的资产准备好用于仿真和操作。机器人交互通过操作反馈和可重放的交互数据闭环生成过程。结果表明，DeformSmith生成的资产在视觉质量和物理合理性方面优于最先进的基线方法，包括PhysGen3D、PhysGM和PhysX-Omni，同时支持为可变形物体的机器人操作合成数据。项目页面：此https URL

英文摘要

Creating deformable assets for robot manipulation requires jointly specifying their geometry, appearance, and physical properties. This is especially challenging for deformable objects, since text and images provide limited evidence about how they deform and respond to contact, yet these responses directly affect their suitability for interaction. Automated generation therefore needs to resolve coupled physical requirements and use interaction evidence to guide construction and refinement. We present DeformSmith, a framework that enables automated generation of interactive, physically credible deformable assets from text or a single image. Through hierarchical agentic construction and a shared physics-grounded harness, it progressively builds, tests, and refines geometry, physical models, material behavior, and robot interaction until the resulting asset is ready for simulation and manipulation. Robot interaction closes the generation loop through manipulation feedback and replayable interaction data. Results show that DeformSmith generates assets with better visual quality and physical plausibility than state-of-the-art baselines, including PhysGen3D, PhysGM, and PhysX-Omni, while supporting the synthesis of data for robotic manipulation of deformable objects. Project page: https://can-lee.github.io/deformsmith-web/

补充信息

备注

Project page: https://can-lee.github.io/deformsmith-web/