arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2510.16136cs.CVcs.AIcs.GR
收藏

GuideFlow3D:用于外观迁移的优化引导校正流

GuideFlow3D: Optimization-Guided Rectified Flow For Appearance Transfer

Stanford University(斯坦福大学);ENPC, IP Paris(巴黎理工学院)

Sayan Deb Sarkar, Sinisa Stekovic, Vincent Lepetit, Iro Armeni

AI总结:

针对图像或文本条件下几何差异较大的3D外观迁移难题,提出免训练的优化引导校正流GuideFlow3D,通过周期性部件感知与自相似性损失引导采样,并以GPT排名和用户研究验证其效果。

摘要:

利用外观对象的不同表示(如图像或文本)将外观迁移到3D资产,因其在游戏、增强现实和数字内容创作等行业中的广泛应用而受到关注。然而,当输入对象与外观对象之间的几何形状存在显著差异时,最先进的方法仍然会失败。一种直接的方法是直接应用3D生成模型,但我们表明,这最终无法产生有吸引力的结果。相反,我们提出了一种受通用引导启发的原则性方法。给定一个以图像或文本为条件的预训练校正流模型,我们的免训练方法通过周期性地添加引导来与采样过程交互。这种引导可以建模为可微损失函数,我们实验了两种不同类型的引导,包括用于外观和自相似性的部件感知损失。实验表明,我们的方法成功地将纹理和几何细节迁移到输入3D资产,在定性和定量上均优于基线。我们还表明,传统指标不适合评估该任务,因为在缺乏真值数据的情况下,它们无法关注局部细节并比较不相似的输入。因此,我们使用基于GPT的系统客观地对输出进行排名,以评估外观迁移质量,确保稳健且类人的评价,我们的用户研究进一步证实了这一点。除展示的场景外,我们的方法具有通用性,可扩展到不同类型的扩散模型和引导函数。

英文摘要:

Transferring appearance to 3D assets using different representations of the appearance object - such as images or text - has garnered interest due to its wide range of applications in industries like gaming, augmented reality, and digital content creation. However, state-of-the-art methods still fail when the geometry between the input and appearance objects is significantly different. A straightforward approach is to directly apply a 3D generative model, but we show that this ultimately fails to produce appealing results. Instead, we propose a principled approach inspired by universal guidance. Given a pretrained rectified flow model conditioned on image or text, our training-free method interacts with the sampling process by periodically adding guidance. This guidance can be modeled as a differentiable loss function, and we experiment with two different types of guidance including part-aware losses for appearance and self-similarity. Our experiments show that our approach successfully transfers texture and geometric details to the input 3D asset, outperforming baselines both qualitatively and quantitatively. We also show that traditional metrics are not suitable for evaluating the task due to their inability of focusing on local details and comparing dissimilar inputs, in absence of ground truth data. We thus evaluate appearance transfer quality with a GPT-based system objectively ranking outputs, ensuring robust and human-like assessment, as further confirmed by our user study. Beyond showcased scenarios, our method is general and could be extended to different types of diffusion models and guidance functions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑