每个像素都有其时刻:通过密集归一化实现超高分辨率非配对图像到图像转换
Every Pixel Has its Moments: Ultra-High-Resolution Unpaired Image-to-Image Translation via Dense Normalization
AI总结:
本文提出密集归一化(DN)层,通过像素级统计矩估计和高效插值与并行策略,在超高分辨率非配对图像到图像转换中同时减少拼接伪影并保持局部颜色对比度,且无需超参数和重新训练。
摘要:
近年来,超高分辨率非配对图像到图像转换的进展旨在通过分块推理来缓解GPU内存有限所带来的约束。然而,现有方法往往在减少明显拼接伪影与保持颜色和色调对比度之间进行折中,这归因于实例归一化层依赖于全局图像级或分块级统计量。在本研究中,我们提出了一种密集归一化(Dense Normalization, DN)层,旨在估计像素级统计矩。该方法有效减少了拼接伪影,同时保持了局部颜色和色调对比度。为应对像素级估计的计算需求,我们进一步提出了一种高效的插值算法。此外,我们设计了一种并行策略,使DN层能够在单次前向传播中运行。通过大量实验,我们证明我们的方法在性能上超越了所有现有方法。值得注意的是,我们的DN层无需超参数,并且可以无缝集成到大多数非配对图像到图像转换框架中,无需重新训练。总体而言,我们的工作为非配对图像到图像转换领域中处理任意分辨率图像的未来探索铺平了道路。代码可在以下地址获取:https://github.com/Kaminyou/Dense-Normalization。
英文摘要:
Recent advancements in ultra-high-resolution unpaired image-to-image translation have aimed to mitigate the constraints imposed by limited GPU memory through patch-wise inference. Nonetheless, existing methods often compromise between the reduction of noticeable tiling artifacts and the preservation of color and hue contrast, attributed to the reliance on global image- or patch-level statistics in the instance normalization layers. In this study, we introduce a Dense Normalization (DN) layer designed to estimate pixel-level statistical moments. This approach effectively diminishes tiling artifacts while concurrently preserving local color and hue contrasts. To address the computational demands of pixel-level estimation, we further propose an efficient interpolation algorithm. Moreover, we invent a parallelism strategy that enables the DN layer to operate in a single pass. Through extensive experiments, we demonstrate that our method surpasses all existing approaches in performance. Notably, our DN layer is hyperparameter-free and can be seamlessly integrated into most unpaired image-to-image translation frameworks without necessitating retraining. Overall, our work paves the way for future exploration in handling images of arbitrary resolutions within the realm of unpaired image-to-image translation. Code is available at: https://github.com/Kaminyou/Dense-Normalization.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。