arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.18623cs.CVcs.GR
收藏

VLMaterial:使用大型视觉-语言模型生成程序化材质

VLMaterial: Procedural Material Generation with Large Vision-Language Models

MIT CSAIL(麻省理工学院计算机科学与人工智能实验室);Columbia University(哥伦比亚大学);ETH Zürich(苏黎世联邦理工学院);Google Research(谷歌研究院)

Beichen Li, Rundi Wu, Armando Solar-Lezama, Changxi Zheng, Liang Shi, Bernd Bickel, Wojciech Matusik

AI总结:

针对由图像重建程序化材质成本高的问题,本文微调 VLM 生成对应 Python 程序,构建开源材质数据集并用 LLM 做程序级增强,在合成与真实数据上均优于既有方法。

摘要:

以功能节点图表示的程序化材质在计算机图形学中无处不在,用于照片级真实感材质外观设计。它们允许用户进行直观且精确的编辑,以获得所需的视觉外观。然而,根据输入图像创建程序化材质需要专业知识和大量精力。在本工作中,我们利用将程序化材质转换为标准 Python 程序的能力,并微调一个大型预训练视觉-语言模型(VLM),以根据输入图像生成此类程序。为实现有效微调,我们还贡献了一个开源程序化材质数据集,并提出通过提示另一个预训练大型语言模型(LLM)来执行程序级增强。通过广泛评估,我们表明我们的方法在合成和真实世界示例上均优于以往方法。

英文摘要:

Procedural materials, represented as functional node graphs, are ubiquitous in computer graphics for photorealistic material appearance design. They allow users to perform intuitive and precise editing to achieve desired visual appearances. However, creating a procedural material given an input image requires professional knowledge and significant effort. In this work, we leverage the ability to convert procedural materials into standard Python programs and fine-tune a large pre-trained vision-language model (VLM) to generate such programs from input images. To enable effective fine-tuning, we also contribute an open-source procedural material dataset and propose to perform program-level augmentation by prompting another pre-trained large language model (LLM). Through extensive evaluation, we show that our method outperforms previous methods on both synthetic and real-world examples.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑