arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2209.06950eess.IVcs.CVcs.LGstat.ML
收藏

基于条件扩散模型的有损图像压缩

Lossy Image Compression with Conditional Diffusion Models

University of California Irvine(加州大学尔湾分校)

Ruihan Yang, Stephan Mandt

AI总结:

本文提出一种基于条件扩散模型的端到端有损图像压缩框架,通过引入“内容”和“纹理”潜在变量实现重建,在FID等感知指标上优于GAN模型,且仅需少量解码步骤即可实现高质量重建。

摘要:

本文概述了一个使用扩散生成模型的端到端优化有损图像压缩框架。该方法依赖于变换编码范式,其中图像被映射到潜在空间进行熵编码,并从那里映射回数据空间进行重建。与基于VAE的神经压缩(其(均值)解码器是确定性神经网络)不同,我们的解码器是条件扩散模型。因此,我们的方法引入了一个额外的“内容”潜在变量,反向扩散过程以此为条件,并使用该变量存储有关图像的信息。表征扩散过程的其余“纹理”变量在解码时被合成。我们表明,模型的性能可以针对感兴趣的感知指标进行调整。我们涉及多个数据集和图像质量评估指标的大量实验表明,我们的方法比基于GAN的模型产生了更强的报告FID分数,同时在几个失真指标上也产生了与基于VAE的模型相当的性能。此外,使用$\mathcal{X}$-parameterization训练扩散模型仅需少量解码步骤即可实现高质量重建,极大地影响了模型的实用性。我们的代码可在以下网址获得:\url{https://github.com/buggyyang/CDC_compression}

英文摘要:

This paper outlines an end-to-end optimized lossy image compression framework using diffusion generative models. The approach relies on the transform coding paradigm, where an image is mapped into a latent space for entropy coding and, from there, mapped back to the data space for reconstruction. In contrast to VAE-based neural compression, where the (mean) decoder is a deterministic neural network, our decoder is a conditional diffusion model. Our approach thus introduces an additional ``content'' latent variable on which the reverse diffusion process is conditioned and uses this variable to store information about the image. The remaining ``texture'' variables characterizing the diffusion process are synthesized at decoding time. We show that the model's performance can be tuned toward perceptual metrics of interest. Our extensive experiments involving multiple datasets and image quality assessment metrics show that our approach yields stronger reported FID scores than the GAN-based model, while also yielding competitive performance with VAE-based models in several distortion metrics. Furthermore, training the diffusion with $\mathcal{X}$-parameterization enables high-quality reconstructions in only a handful of decoding steps, greatly affecting the model's practicality. Our code is available at: \url{https://github.com/buggyyang/CDC_compression}

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑