从人类感知视角出发，基于LMMs语义解耦的极低码率图像压缩
Extremely low-bitrate Image Compression Semantically Disentangled by LMMs from a Human Perception Perspective
AI总结：
提出SEDIC框架，利用LMMs提取语义组件并基于预训练ControlNet的ORAG模型逐物体渐进恢复，实现极低码率下高感知质量和语义一致性的图像压缩。
摘要：
在极低码率下实现图像压缩，同时兼顾语义一致性和高感知质量，仍然是一个重大挑战。受人类渐进式感知机制的启发，本文提出了一种语义解耦图像压缩框架（SEDIC）。首先，通过一个学习型图像编码器获得一张极度压缩的参考图像。然后，利用LMMs提取关键语义组件，包括整体描述、物体详细描述和语义分割掩码。我们提出了一种基于预训练ControlNet构建的、无需训练的、带有注意力引导的物体恢复模型（ORAG），以在物体级文本描述和语义掩码的条件下恢复物体细节。基于所提出的ORAG，我们设计了一个多阶段语义图像解码器，从极度压缩的参考图像开始，逐物体地渐进恢复细节，最终生成高质量、高保真的重建图像。实验结果表明，SEDIC显著优于最先进的方法，在极低码率（≤0.05 bpp）下实现了卓越的感知质量和语义一致性。
英文摘要：
It remains a significant challenge to compress images at extremely low bitrate while achieving both semantic consistency and high perceptual quality. Inspired by human progressive perception mechanism, we propose a Semantically Disentangled Image Compression framework (SEDIC) in this paper. Initially, an extremely compressed reference image is obtained through a learned image encoder. Then we leverage LMMs to extract essential semantic components, including overall descriptions, object detailed description, and semantic segmentation masks. We propose a training-free Object Restoration model with Attention Guidance (ORAG) built on pre-trained ControlNet to restore object details conditioned by object-level text descriptions and semantic masks. Based on the proposed ORAG, we design a multistage semantic image decoder to progressively restore the details object by object, starting from the extremely compressed reference image, ultimately generating high-quality and high-fidelity reconstructions. Experimental results demonstrate that SEDIC significantly outperforms state-of-the-art approaches, achieving superior perceptual quality and semantic consistency at extremely low-bitrates ($\le$ 0.05 bpp).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
