arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2312.02647cs.CV
收藏

TPA3D:用于快速文本到3D生成的三平面注意力

TPA3D: Triplane Attention for Fast Text-to-3D Generation

National Taiwan University(台湾大学);NVIDIA(英伟达)

Bin-Shih Wu, Hong-En Chen, Sheng-Yu Huang, Yu-Chiang Frank Wang

AI总结:

本文提出TPA3D,一种基于GAN的端到端可训练模型,通过在句子级和词级文本特征上引入三平面注意力机制,实现快速且与细粒度描述一致的文本到3D生成。

摘要:

由于缺乏大规模的文本-3D对应数据,近期的文本到3D生成工作主要依赖利用2D扩散模型来合成3D数据。由于基于扩散的方法通常需要大量的训练和推理优化时间,使用基于GAN的模型对于快速3D生成仍然具有吸引力。在这项工作中,我们提出了用于文本引导3D生成的三平面注意力(TPA3D),这是一个端到端可训练的基于GAN的深度学习模型,用于快速文本到3D生成。在训练过程中仅观察3D形状数据及其渲染的2D图像,我们的TPA3D被设计为检索详细的视觉描述,以合成相应的3D网格数据。这是通过在提取的句子级和词级文本特征上提出的注意力机制实现的。在我们的实验中,我们表明TPA3D生成了与细粒度描述一致的高质量3D纹理形状,同时可以观察到令人印象深刻的计算效率。

英文摘要:

Due to the lack of large-scale text-3D correspondence data, recent text-to-3D generation works mainly rely on utilizing 2D diffusion models for synthesizing 3D data. Since diffusion-based methods typically require significant optimization time for both training and inference, the use of GAN-based models would still be desirable for fast 3D generation. In this work, we propose Triplane Attention for text-guided 3D generation (TPA3D), an end-to-end trainable GAN-based deep learning model for fast text-to-3D generation. With only 3D shape data and their rendered 2D images observed during training, our TPA3D is designed to retrieve detailed visual descriptions for synthesizing the corresponding 3D mesh data. This is achieved by the proposed attention mechanisms on the extracted sentence and word-level text features. In our experiments, we show that TPA3D generates high-quality 3D textured shapes aligned with fine-grained descriptions, while impressive computation efficiency can be observed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑