arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2205.02087cs.CVcs.LG
收藏

超复杂图像到图像翻译

Hypercomplex Image-to-Image Translation

Sapienza University of Rome(罗马大学)

Eleonora Grassucci, Luigi Sigillo, Aurelio Uncini, Danilo Comminiello

摘要:

图像到图像翻译(I2I)旨在将内容表示从输入域转移到输出域,沿着不同的目标域跳跃。最近的I2I生成模型在这项任务中取得了出色的成绩,包括一组不同的深度网络,每个网络都有数千万个参数。此外,图像通常是由RGB通道组成的三维图像,并且常见的神经模型没有考虑维度相关性,从而丢失了有益的信息。在本文中,我们建议利用超复杂代数属性来定义轻量级I2I生成模型,该模型能够保留图像尺寸之间的预先存在的关系,从而利用额外的输入信息。在多个I2I基准测试中,我们展示了所提出的四元数StarGANv2和参数化超复杂StarGANv2(PHStarGANv2)如何减少参数和存储内存量,同时确保高域转换性能和良好的图像质量,如FID和LPIPS分数所测量的。完整代码可在以下网址获得:https://github.com/ispamm/HI2I。

英文摘要:

Image-to-image translation (I2I) aims at transferring the content representation from an input domain to an output one, bouncing along different target domains. Recent I2I generative models, which gain outstanding results in this task, comprise a set of diverse deep networks each with tens of million parameters. Moreover, images are usually three-dimensional being composed of RGB channels and common neural models do not take dimensions correlation into account, losing beneficial information. In this paper, we propose to leverage hypercomplex algebra properties to define lightweight I2I generative models capable of preserving pre-existing relations among image dimensions, thus exploiting additional input information. On manifold I2I benchmarks, we show how the proposed Quaternion StarGANv2 and parameterized hypercomplex StarGANv2 (PHStarGANv2) reduce parameters and storage memory amount while ensuring high domain translation performance and good image quality as measured by FID and LPIPS scores. Full code is available at: https://github.com/ispamm/HI2I.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑