arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2006.11337cs.CV
收藏

图像情感传递

Image Sentiment Transfer

University of Rochester(罗切斯特大学)

Tianlang Chen, Wei Xiong, Haitian Zheng, Jiebo Luo

摘要:

在这项工作中,我们介绍了一个重要的,但仍然是未开发的研究任务-图像情感转移。与其他已经得到充分研究的相关任务相比,例如图像到图像的翻译和图像风格转移,转移图像的情感更具挑战性。给定输入图像,传递每个包含对象的情感的规则可能完全不同,使得通过单个参考图像执行全局图像传递的现有方法不足以实现令人满意的性能。在本文中,我们提出了一个有效和灵活的框架,在对象级别执行图像情感转移。该方法首先检测目标并提取其像素级模板,然后在多幅参考图像的引导下对目标进行目标级情感传递。对于核心的对象级情感传递,我们提出了一种新的情感感知GAN(SentiGAN)。对SentiGAN进行全局图像级和局部对象级的监督训练。更重要的是,应用与内容对齐步骤协作的有效内容解缠损失以更好地解缠输入图像的剩余情感相关信息。大量的定量和定性实验进行面向对象的VSO数据集,我们创建,证明了所提出的框架的有效性。

英文摘要:

In this work, we introduce an important but still unexplored research task -- image sentiment transfer. Compared with other related tasks that have been well-studied, such as image-to-image translation and image style transfer, transferring the sentiment of an image is more challenging. Given an input image, the rule to transfer the sentiment of each contained object can be completely different, making existing approaches that perform global image transfer by a single reference image inadequate to achieve satisfactory performance. In this paper, we propose an effective and flexible framework that performs image sentiment transfer at the object level. It first detects the objects and extracts their pixel-level masks, and then performs object-level sentiment transfer guided by multiple reference images for the corresponding objects. For the core object-level sentiment transfer, we propose a novel Sentiment-aware GAN (SentiGAN). Both global image-level and local object-level supervisions are imposed to train SentiGAN. More importantly, an effective content disentanglement loss cooperating with a content alignment step is applied to better disentangle the residual sentiment-related information of the input image. Extensive quantitative and qualitative experiments are performed on the object-oriented VSO dataset we create, demonstrating the effectiveness of the proposed framework.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑