arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1810.09718cs.GR
收藏

使用渲染感知深度网络捕获单个图像SVBRDF

Single-Image SVBRDF Capture with a Rendering-Aware Deep Network

Valentin Deschaintre, Miika Aittala, Fredo Durand, George Drettakis, Adrien Bousseau

摘要:

纹理、高光和阴影是许多视觉线索中的一些,这些视觉线索允许人类在单个图片中感知材料外观。然而,恢复空间变化的双向反射分布函数(SVBRDFs)从一个单一的图像的基础上,这样的线索在计算机图形学的研究人员提出了几十年的挑战。我们通过训练一个深度神经网络来自动提取和理解这些视觉线索,从而解决轻量级外观捕捉问题。一旦经过训练,我们的网络就能够从手持闪光灯照亮的平面的单张图片中恢复每像素的正常,漫射反射,镜面反射和镜面粗糙度。我们通过在训练数据获取和网络设计方面引入几项创新来实现这一目标。为了训练,我们利用了一个由艺术家创建的大型程序SVBRDF数据集,我们在多个照明方向下进行采样和渲染。我们通过材料混合来进一步放大数据,以覆盖各种各样的阴影效果,这使得我们的网络可以跨许多材料类别工作。材料样本的远距离区域通常提供互补的视觉线索,我们设计了一个网络,该网络将用于局部特征提取的编码器-解码器卷积轨道与用于全局特征提取和传播的全连接轨道相结合。许多重要的材质效果都是依赖于视图的,因此在单个图像中观察时是模糊的。我们通过将损失定义为可区分的SVBRDF相似性度量来应对这一挑战,该度量将预测地图的渲染与来自多个照明和观看方向的地面实况渲染进行比较。结合在一起,这些新的成分带来了明显的改进,超过了现有技术的方法,用于单次拍摄捕捉空间变化的BRDF。

英文摘要:

Texture, highlights, and shading are some of many visual cues that allow humans to perceive material appearance in single pictures. Yet, recovering spatially-varying bi-directional reflectance distribution functions (SVBRDFs) from a single image based on such cues has challenged researchers in computer graphics for decades. We tackle lightweight appearance capture by training a deep neural network to automatically extract and make sense of these visual cues. Once trained, our network is capable of recovering per-pixel normal, diffuse albedo, specular albedo and specular roughness from a single picture of a flat surface lit by a hand-held flash. We achieve this goal by introducing several innovations on training data acquisition and network design. For training, we leverage a large dataset of artist-created, procedural SVBRDFs which we sample and render under multiple lighting directions. We further amplify the data by material mixing to cover a wide diversity of shading effects, which allows our network to work across many material classes. Motivated by the observation that distant regions of a material sample often offer complementary visual cues, we design a network that combines an encoder-decoder convolutional track for local feature extraction with a fully-connected track for global feature extraction and propagation. Many important material effects are view-dependent, and as such ambiguous when observed in a single image. We tackle this challenge by defining the loss as a differentiable SVBRDF similarity metric that compares the renderings of the predicted maps against renderings of the ground truth from several lighting and viewing directions. Combined together, these novel ingredients bring clear improvement over state of the art methods for single-shot capture of spatially varying BRDFs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑