arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2201.06415cs.CVeess.IV
收藏

通过在潜在分割空间中注入噪声来提高语义分割CycleGANs的性能

Improving Performance of Semantic Segmentation CycleGANs by Noise Injection into the Latent Segmentation Space

Technische Universität Braunschweig(布伦瑞克工业大学)

Jonas Löhdefink, Tim Fingscheidt

摘要:

近年来,语义分割得到了计算机视觉领域的广泛应用。受非常通用的CycleGAN架构的启发,我们将语义分割与循环一致性的概念相结合,以实现多任务训练协议。然而,学习在很大程度上被所谓的隐写效应所阻止,隐写效应在潜在的分割域中表现为水印,使图像重建成为一项过于简单的任务。为了解决这个问题,我们提出了一个噪声注入,基于量化噪声或高斯噪声,以避免这种不利的信息流的循环架构。我们发现,噪声注入显着减少水印的生成,从而允许识别高度相关的类,如“交通标志”,这是很难检测到的ERFNet基线。我们报告mIoU和PSNR结果的城市景观数据集的语义分割和图像重建,分别。所提出的方法允许在没有噪声注入的情况下,在相同的CycleGAN上实现Cityscapes验证集5.7%的绝对mIoU改进,并且在ERFNet非循环基线上仍然是绝对4.9%。

英文摘要:

In recent years, semantic segmentation has taken benefit from various works in computer vision. Inspired by the very versatile CycleGAN architecture, we combine semantic segmentation with the concept of cycle consistency to enable a multitask training protocol. However, learning is largely prevented by the so-called steganography effect, which expresses itself as watermarks in the latent segmentation domain, making image reconstruction a too easy task. To combat this, we propose a noise injection, based either on quantization noise or on Gaussian noise addition to avoid this disadvantageous information flow in the cycle architecture. We find that noise injection significantly reduces the generation of watermarks and thus allows the recognition of highly relevant classes such as "traffic signs", which are hardly detected by the ERFNet baseline. We report mIoU and PSNR results on the Cityscapes dataset for semantic segmentation and image reconstruction, respectively. The proposed methodology allows to achieve an mIoU improvement on the Cityscapes validation set of 5.7% absolute over the same CycleGAN without noise injection, and still an absolute 4.9% over the ERFNet non-cyclic baseline.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑