arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2307.01944cs.LGcs.CVcs.ITmath.IT
收藏

文本+草图:超低码率下的图像压缩

Text + Sketch: Image Compression at Ultra Low Rates

University of Pennsylvania(宾夕法尼亚大学)

Eric Lei, Yiğit Berkay Uslu, Hamed Hassani, Shirin Saeedi Bidokhti

AI总结:

本文提出利用预训练文本到图像生成模型,结合文本描述与边信息,在极低码率下实现保留语义和空间结构的高保真图像压缩,无需端到端训练即可显著优于现有学习型压缩器。

摘要:

文本到图像生成模型的最新进展使得从简短文本描述生成高质量图像成为可能。这些基础模型在十亿级数据集上预训练后,无需进一步训练或仅需极少训练即可有效完成各种下游任务。一个自然的问题是:如何将这些模型用于图像压缩。我们研究了几种技术,可直接利用预训练模型实现面向新型低码率场景的压缩方案。我们展示了如何将文本描述与边信息结合使用,生成既保留原始图像语义又保留其空间结构的高保真重建结果。我们证明,在极低比特率下,尽管没有进行端到端训练,我们的方法在感知保真度和语义保真度方面都能显著优于现有的学习型压缩器。

英文摘要:

Recent advances in text-to-image generative models provide the ability to generate high-quality images from short text descriptions. These foundation models, when pre-trained on billion-scale datasets, are effective for various downstream tasks with little or no further training. A natural question to ask is how such models may be adapted for image compression. We investigate several techniques in which the pre-trained models can be directly used to implement compression schemes targeting novel low rate regimes. We show how text descriptions can be used in conjunction with side information to generate high-fidelity reconstructions that preserve both semantics and spatial structure of the original. We demonstrate that at very low bit-rates, our method can significantly improve upon learned compressors in terms of perceptual and semantic fidelity, despite no end-to-end training.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑