arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2405.09431cs.CVcs.GR
收藏

野外文本到3D内容生成综述

A Survey On Text-to-3D Contents Generation In The Wild

The Hong Kong University of Science and Technology(香港科技大学)

Chenhan Jiang

AI总结:

本综述针对野外文本到3D生成技术,梳理背景、3D表示形式,分类对比生成流水线方法,分析其优缺点,并指出未来研究方向,以推动开放词汇文本条件3D内容创建的发展。

摘要:

3D内容创建在游戏、机器人仿真、虚拟现实等诸多应用中发挥着关键作用。然而,这一过程劳动强度大、耗时长,需要熟练设计师投入大量精力才能创建单个3D资产。为应对这一挑战,文本到3D生成技术应运而生,成为实现3D创建自动化的可行方案。这类技术借助大视觉语言模型的成功成果,旨在根据文本描述生成3D内容。尽管该领域近期取得了一定进展,但现有方案在生成质量和效率方面仍存在明显局限。本综述对最新的文本到3D创建方法展开了深入调研。我们全面介绍了文本到3D创建的背景知识,包括对训练所用数据集及评估生成3D模型质量所用指标的讨论。随后,我们深入探究了作为3D生成过程基础的各类3D表示形式。此外,我们对快速增长的生成流水线相关文献进行了全面对比,将其划分为前馈生成器、基于优化的生成以及视图重建方法三类。通过分析这些方法的优缺点,我们旨在阐明它们各自的能力与局限。最后,我们指出了若干未来研究的可行方向。我们希望本综述能进一步激励研究人员探索开放词汇文本条件3D内容创建的潜力。

英文摘要:

3D content creation plays a vital role in various applications, such as gaming, robotics simulation, and virtual reality. However, the process is labor-intensive and time-consuming, requiring skilled designers to invest considerable effort in creating a single 3D asset. To address this challenge, text-to-3D generation technologies have emerged as a promising solution for automating 3D creation. Leveraging the success of large vision language models, these techniques aim to generate 3D content based on textual descriptions. Despite recent advancements in this area, existing solutions still face significant limitations in terms of generation quality and efficiency. In this survey, we conduct an in-depth investigation of the latest text-to-3D creation methods. We provide a comprehensive background on text-to-3D creation, including discussions on datasets employed in training and evaluation metrics used to assess the quality of generated 3D models. Then, we delve into the various 3D representations that serve as the foundation for the 3D generation process. Furthermore, we present a thorough comparison of the rapidly growing literature on generative pipelines, categorizing them into feedforward generators, optimization-based generation, and view reconstruction approaches. By examining the strengths and weaknesses of these methods, we aim to shed light on their respective capabilities and limitations. Lastly, we point out several promising avenues for future research. With this survey, we hope to inspire researchers further to explore the potential of open-vocabulary text-conditioned 3D content creation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑