arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2303.15764cs.CV
收藏

X-Mesh:基于动态文本引导的快速准确文本驱动3D风格化

X-Mesh: Towards Fast and Accurate Text-driven 3D Stylization via Dynamic Textual Guidance

Xiamen University(厦门大学);Contemporary Amperex Technology Co., Limited (CATL)(宁德时代新能源科技股份有限公司(CATL))

Yiwei Ma, Xiaioqing Zhang, Xiaoshuai Sun, Jiayi Ji, Haowei Wang, Guannan Jiang, Weilin Zhuang, Rongrong Ji

AI总结:

提出X-Mesh框架,通过文本引导动态注意力模块(TDAM)在顶点特征提取时动态整合目标文本指导,实现更准确、更快的3D风格化,并引入MIT-30基准与自动化指标解决评估问题。

摘要:

文本驱动的3D风格化是计算机视觉(CV)和计算机图形学(CG)领域中一项复杂且关键的任务,旨在将裸网格转换为符合目标文本的样式。先前的方法采用与文本无关的多层感知器(MLP),在CLIP损失的监督下预测目标网格的属性。然而,这种与文本无关的架构在预测属性时缺乏文本指导,从而导致风格化效果不理想和收敛速度缓慢。为了解决这些局限性,我们提出了X-Mesh,一个创新的文本驱动3D风格化框架,它包含一个新颖的文本引导动态注意力模块(TDAM)。TDAM在顶点特征提取过程中,通过利用与文本相关的空间和通道注意力,动态整合目标文本的指导,从而实现更准确的属性预测和更快的收敛速度。此外,现有工作缺乏标准基准和自动化评估指标,通常依赖主观且不可复现的用户研究来评估风格化3D资产的质量。为了克服这一局限,我们引入了一个新的标准文本-网格基准,即MIT-30,以及两个自动化指标,这将使未来的研究能够实现公平和客观的比较。我们的大量定性和定量实验表明,X-Mesh优于以前的最先进方法。

英文摘要:

Text-driven 3D stylization is a complex and crucial task in the fields of computer vision (CV) and computer graphics (CG), aimed at transforming a bare mesh to fit a target text. Prior methods adopt text-independent multilayer perceptrons (MLPs) to predict the attributes of the target mesh with the supervision of CLIP loss. However, such text-independent architecture lacks textual guidance during predicting attributes, thus leading to unsatisfactory stylization and slow convergence. To address these limitations, we present X-Mesh, an innovative text-driven 3D stylization framework that incorporates a novel Text-guided Dynamic Attention Module (TDAM). The TDAM dynamically integrates the guidance of the target text by utilizing text-relevant spatial and channel-wise attentions during vertex feature extraction, resulting in more accurate attribute prediction and faster convergence speed. Furthermore, existing works lack standard benchmarks and automated metrics for evaluation, often relying on subjective and non-reproducible user studies to assess the quality of stylized 3D assets. To overcome this limitation, we introduce a new standard text-mesh benchmark, namely MIT-30, and two automated metrics, which will enable future research to achieve fair and objective comparisons. Our extensive qualitative and quantitative experiments demonstrate that X-Mesh outperforms previous state-of-the-art methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑