arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2009.08255cs.CV
收藏

基于辅助照明的对抗性图像合成

Adversarial Image Composition with Auxiliary Illumination

Nanyang Technological University(南洋理工大学);Alibaba DAMO Academy(阿里巴巴达摩院)

Fangneng Zhan, Shijian Lu, Changgong Zhang, Feiying Ma, Xuansong Xie

摘要:

处理前景物体和背景图像之间的不一致性是高保真图像合成中具有挑战性的任务。现有技术的方法努力通过使前景对象的风格适应于与背景图像兼容来协调合成图像,而合成图像内的前景对象的潜在阴影对合成真实性至关重要,这在很大程度上被忽略。在本文中,我们提出了一个对抗性的图像合成网络(AIC-Net),通过考虑潜在的阴影,前景对象的项目在合成图像中实现逼真的图像合成。提出了一种新的分支生成机制,将阴影的生成和前景样式的转换分离开来,使这两个任务同时得到最优的完成。设计了可微空间变换模块,将局部协调与全局协调有效地结合起来,实现了局部协调与全局协调的联合优化。对行人和汽车组成任务的大量实验表明,所提出的AIC-Net在定性和定量方面都取得了优异的组成性能。

英文摘要:

Dealing with the inconsistency between a foreground object and a background image is a challenging task in high-fidelity image composition. State-of-the-art methods strive to harmonize the composed image by adapting the style of foreground objects to be compatible with the background image, whereas the potential shadow of foreground objects within the composed image which is critical to the composition realism is largely neglected. In this paper, we propose an Adversarial Image Composition Net (AIC-Net) that achieves realistic image composition by considering potential shadows that the foreground object projects in the composed image. A novel branched generation mechanism is proposed, which disentangles the generation of shadows and the transfer of foreground styles for optimal accomplishment of the two tasks simultaneously. A differentiable spatial transformation module is designed which bridges the local harmonization and the global harmonization to achieve their joint optimization effectively. Extensive experiments on pedestrian and car composition tasks show that the proposed AIC-Net achieves superior composition performance qualitatively and quantitatively.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑