使用感知属性进行自然材质外观编辑
In-the-wild Material Appearance Editing using Perceptual Attributes
摘要:
鉴于光与物质之间相互作用的复杂性以及人类感知的矛盾性,从单个图像中直观地编辑材料的外观是一项具有挑战性的任务。传统上,通过估计场景的附加因素(如几何形状或照明)来解决这个问题,从而解决逆渲染问题并将结果的最终质量与这些估计的质量相匹配。我们提出了一个单图像外观编辑框架,该框架允许我们通过增加或减少描述这种外观的高级感知属性(例如,光泽的或金属的)。我们的框架将单个对象的野外图像作为输入,其中几何形状,材料和照明不受控制,并且不需要逆渲染。我们依赖于生成模型,并设计了一种具有选择性传输单元(STU)单元的新型架构,该单元允许在编辑后的图像中保留输入图像的高频细节。为了训练我们的框架,我们利用一个数据集,该数据集具有用基于物理的算法渲染的成对合成图像,以及相应的高级别感知属性的众包评级。我们展示了我们的材料编辑框架优于最先进的技术,并展示了其在合成图像,野外真实世界照片和视频序列上的适用性。
英文摘要:
Intuitively editing the appearance of materials from a single image is a challenging task given the complexity of the interactions between light and matter, and the ambivalence of human perception. This problem has been traditionally addressed by estimating additional factors of the scene like geometry or illumination, thus solving an inverse rendering problem and subduing the final quality of the results to the quality of these estimations. We present a single-image appearance editing framework that allows us to intuitively modify the material appearance of an object by increasing or decreasing high-level perceptual attributes describing such appearance (e.g., glossy or metallic). Our framework takes as input an in-the-wild image of a single object, where geometry, material, and illumination are not controlled, and inverse rendering is not required. We rely on generative models and devise a novel architecture with Selective Transfer Unit (STU) cells that allow to preserve the high-frequency details from the input image in the edited one. To train our framework we leverage a dataset with pairs of synthetic images rendered with physically-based algorithms, and the corresponding crowd-sourced ratings of high-level perceptual attributes. We show that our material editing framework outperforms the state of the art, and showcase its applicability on synthetic images, in-the-wild real-world photographs, and video sequences.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。