野外照片集的联合材质和光照估计
Joint Material and Illumination Estimation from Photo Sets in the Wild
摘要：
2D互联网图像中的形状、材料和照明的忠实操纵将极大地受益于将外观可靠地分解为材料（即，漫射和镜面反射）和照明（即，环境地图）。一方面，产生非常高保真度结果的当前方法通常需要受控设置、昂贵的设备或显著的人工努力。另一方面，自动化和工作在“野外”互联网图像上的方法通常只提取低频照明或漫射材料。在这项工作中，我们建议利用一组照片，以共同估计的非漫反射材料和尖锐的照明在一个不受控制的设置。我们的关键观察是，在不同的照明下看到同一材料的多个实例（即，环境），并且相同照明下的不同材料提供了可以被利用以产生高质量解决方案的有价值的约束（即，镜面材质和环境照明）。当在单一环境中观察多种材料或在多个环境中观察单一材料时，也会出现类似的约束。这种方法的核心是一个优化过程，它使用两个神经网络，这两个神经网络是在合成图像上训练的，以预测参数空间中的良好梯度，从而观察到反射光。我们在一系列合成和真实示例上评估我们的方法，以生成高质量的估计值，通过用户研究将我们的结果与最先进的替代方案进行定性比较，并展示照片一致的图像处理，否则很难实现。
英文摘要：
Faithful manipulation of shape, material, and illumination in 2D Internet images would greatly benefit from a reliable factorization of appearance into material (i.e., diffuse and specular) and illumination (i.e., environment maps). On the one hand, current methods that produce very high fidelity results, typically require controlled settings, expensive devices, or significant manual effort. To the other hand, methods that are automatic and work on 'in the wild' Internet images, often extract only low-frequency lighting or diffuse materials. In this work, we propose to make use of a set of photographs in order to jointly estimate the non-diffuse materials and sharp lighting in an uncontrolled setting. Our key observation is that seeing multiple instances of the same material under different illumination (i.e., environment), and different materials under the same illumination provide valuable constraints that can be exploited to yield a high-quality solution (i.e., specular materials and environment illumination) for all the observed materials and environments. Similar constraints also arise when observing multiple materials in a single environment, or a single material across multiple environments. The core of this approach is an optimization procedure that uses two neural networks that are trained on synthetic images to predict good gradients in parametric space given observation of reflected light. We evaluate our method on a range of synthetic and real examples to generate high-quality estimates, qualitatively compare our results against state-of-the-art alternatives via a user study, and demonstrate photo-consistent image manipulation that is otherwise very challenging to achieve.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
