arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 1903.11248cs.CV
收藏

模拟摄像机内的颜色流水线进行摄像机感知的对象合成

Mimicking the In-Camera Color Pipeline for Camera-Aware Object Compositing

University of Toronto(多伦多大学);Vector Institute(向量研究所);University of Science and Technology of China(中国科学技术大学);Microsoft Research(微软研究院);Peking University(北京大学);NVIDIA(英伟达)

Jun Gao, Xiao Li, Liwei Wang, Sanja Fidler, Stephen Lin

摘要:

我们提出了一种用于将虚拟对象合成到照片中的方法,使得对象颜色看起来已经由照片的相机成像管道处理。以这种相机感知的方式进行合成对于高真实感至关重要,并且它需要推断照片管道中的颜色变换,由于从场景到照片存在固有的一对多映射,这是具有挑战性的。为了解决这个问题的情况下,从一个未知的相机拍摄的一张照片,我们提出了一个双重学习的方法,其中反向颜色变换(从照片到场景)联合估计。反向变换的学习被用来促进正向映射的学习,通过强制执行两个过程的循环一致性。我们还采用了一个特征共享模式,从反向映射中的目标照片中提取证据,以指导正向颜色变换。我们的双重学习方法实现了对象合成结果,超过了替代技术。

英文摘要:

We present a method for compositing virtual objects into a photograph such that the object colors appear to have been processed by the photo's camera imaging pipeline. Compositing in such a camera-aware manner is essential for high realism, and it requires the color transformation in the photo's pipeline to be inferred, which is challenging due to the inherent one-to-many mapping that exists from a scene to a photo. To address this problem for the case of a single photo taken from an unknown camera, we propose a dual-learning approach in which the reverse color transformation (from the photo to the scene) is jointly estimated. Learning of the reverse transformation is used to facilitate learning of the forward mapping, by enforcing cycle consistency of the two processes. We additionally employ a feature sharing schema to extract evidence from the target photo in the reverse mapping to guide the forward color transformation. Our dual-learning approach achieves object compositing results that surpass those of alternative techniques.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑