arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2301.06015cs.CV
收藏

3D场景中基于扩散的生成、优化和规划

Diffusion-based Generation, Optimization, and Planning in 3D Scenes

National Key Laboratory of General Artificial Intelligence, BIGAI(北京通用人工智能研究院通用人工智能国家重点实验室);Tsinghua University(清华大学);Peking University(北京大学);Beijing Institute of Technology(北京理工大学)

Siyuan Huang, Zan Wang, Puhao Li, Baoxiong Jia, Tengyu Liu, Yixin Zhu, Wei Liang, Song-Chun Zhu

摘要:

介绍了一种用于三维场景理解的条件生成模型SceneDiffuser。SceneDiffuser为解决场景条件生成、优化和规划提供了一个统一的模型。与以前的作品相比,SceneDiffuser本质上是场景感知的,基于物理的,面向目标的。通过迭代采样策略,SceneDiffuser以完全可区分的方式通过基于扩散的去噪过程联合制定场景感知生成、基于物理的优化和面向目标的规划。这样的设计消除了不同模块之间的差异和先前场景条件生成模型的后崩溃。我们评估SceneDiffuser与各种3D场景理解任务,包括人体姿势和运动生成,灵巧把握生成,3D导航的路径规划,和机器人手臂的运动规划。结果表明,与以前的模型相比,显着的改进,展示了巨大的潜力,SceneDiffuser为广大社区的三维场景理解。

英文摘要:

We introduce SceneDiffuser, a conditional generative model for 3D scene understanding. SceneDiffuser provides a unified model for solving scene-conditioned generation, optimization, and planning. In contrast to prior works, SceneDiffuser is intrinsically scene-aware, physics-based, and goal-oriented. With an iterative sampling strategy, SceneDiffuser jointly formulates the scene-aware generation, physics-based optimization, and goal-oriented planning via a diffusion-based denoising process in a fully differentiable fashion. Such a design alleviates the discrepancies among different modules and the posterior collapse of previous scene-conditioned generative models. We evaluate SceneDiffuser with various 3D scene understanding tasks, including human pose and motion generation, dexterous grasp generation, path planning for 3D navigation, and motion planning for robot arms. The results show significant improvements compared with previous models, demonstrating the tremendous potential of SceneDiffuser for the broad community of 3D scene understanding.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑