arXiv 2509.221512026-05-13cs.CVcs.CL

MultiMat: 多模态程序合成用于基于过程的材料生成

MultiMat: Multimodal Program Synthesis for Procedural Materials using Large Multimodal Models

University of Mannheim（曼海姆大学）；Adobe Research（Adobe研究）

Jonas Belouadi, Tamy Boubekeur, Adrien Kaiser

备注：: Accepted at ICLR 2026 (poster)

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

MultiMat利用大规模多模态模型实现多模态程序合成，提升生成过程材料图的效率与视觉质量，优于纯文本基线方法。

摘要：

材料节点图是生成程序化材料的2D通道的程序，包括几何如粗糙度和位移图，以及反射率如albedo和导电性图。它们在计算机图形学中对于以参数化和任意分辨率表示虚拟3D对象的外观至关重要。特别是，它们的有向无环图结构和中间状态使交互式外观建模能够实现模块化和可解释的工作流程。然而，创建此类图仍然具有挑战性，通常需要专业培训。尽管最近的神经程序合成方法试图简化这一过程，但它们仅将图表示为文本程序，无法捕捉到节点图本质上视觉-空间性质，这使得它们对人类易于理解。为了解决这一差距，我们提出了MultiMat，一种多模态程序合成框架，利用大型多模态模型来处理视觉和文本图表示，以提高程序化材料图的生成效果。我们训练我们的模型在一个新的生产质量程序化材料数据集上，并将其与一种受约束的树搜索推理算法结合，该算法确保静态正确性的同时，能够高效地在程序空间中导航。我们的实验结果表明，我们的多模态程序合成方法在无条件和有条件图合成中比纯文本基线更高效，具有更高的视觉质量和保真度，建立了新的最先进性能。

英文摘要：

Material node graphs are programs that generate the 2D channels of procedural materials, including geometry such as roughness and displacement maps, and reflectance such as albedo and conductivity maps. They are essential in computer graphics for representing the appearance of virtual 3D objects parametrically and at arbitrary resolution. In particular, their directed acyclic graph structure and intermediate states enable a modular, interpretable workflow for interactive appearance modeling. However, creating such graphs remains challenging and typically requires professional training. While recent neural program synthesis approaches attempt to simplify this process, they solely represent graphs as textual programs, failing to capture the inherently visual-spatial nature of node graphs that makes them accessible to humans. To address this gap, we present MultiMat, a multimodal program synthesis framework that leverages large multimodal models to process both visual and textual graph representations for improved generation of procedural material graphs. We train our models on a new dataset of production-quality procedural materials and combine them with a constrained tree search inference algorithm that ensures static correctness while efficiently navigating the program space. Our experimental results show that our multimodal program synthesis method is more efficient in both unconditional and conditional graph synthesis with higher visual quality and fidelity than text-only baselines, establishing new state-of-the-art performance.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

AI总结：

摘要：

英文摘要：

相关论文