发表机构
NVIDIA; NTU; PKU（英伟达; 南洋理工大学; 北京大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
USDCraft是基于LLM的程序化建模框架，通过源几何分析与迭代几何复检，生成可直接用于Isaac Sim的关节式USD资产，在关节结构恢复基准中表现领先，可支撑真实到仿真再到真实的机器人操纵。
AI 中文摘要
几何保真且具备功能的关节式3D资产对于真实到仿真的机器人操纵至关重要，在仿真中训练的策略必须能迁移到物理对象上。近期基于网格的方法从带标注的3D资产中学习推断关节结构，但当现实世界对象超出训练分布、或其网格不完整或损坏时，部署仍具挑战性。为解决这些局限，我们将关节式资产重构建模为基于部分几何证据的程序化建模，并引入USDCraft框架，该框架中预训练的大语言模型（LLM）无需特定任务训练即可编写和修订适用于仿真的关节式资产的可执行程序。我们提出源几何分析，将源网格转换为度量文本描述，区分观测表面与未知空间；还提出迭代几何复检，以相同表示重新编码每个候选结果，使差异指向程序编辑，未观测区域则留待补全。视觉反馈与物理创作指导完善建模过程，最终生成带显式物理属性的关节式USD资产，无需手动调整即可加载到Isaac Sim中。实验表明，USDCraft在两个基准上实现了领先的关节结构恢复效果，并验证了其在真实到仿真再到真实的机器人操纵中的有效性。
英文摘要
Geometrically faithful and functional articulated 3D assets are essential for real-to-sim robot manipulation, where policies trained in simulation must transfer to physical objects. Recent mesh-based methods learn to infer articulation from annotated 3D assets, but deployment remains challenging when real-world objects fall outside the training distribution or their meshes are incomplete or corrupted. To address these limitations, we formulate articulated asset reconstruction as programmatic modeling grounded in partial geometric evidence and introduce USDCraft, a framework in which a pretrained LLM writes and revises executable programs for simulation-ready articulated assets without task-specific training. We propose source geometry analysis, which converts the source mesh into a metric textual description that distinguishes observed surface from unknown space, and iterative geometric rechecking, which re-encodes each candidate in the same representation so that discrepancies point to program edits while unobserved regions remain open to completion. Visual feedback and physical authoring guidance complete the modeling process, which produces articulated USD assets with explicit physical properties that load into Isaac Sim without manual adjustment. Experiments demonstrate leading articulation recovery on two benchmarks and validate USDCraft's effectiveness for real-to-sim-to-real robot manipulation.
CommentsProject page: https://xingyoujun.github.io/usdcraft
