arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.18501cs.CVcs.AI
收藏

高效视频扩散与稀疏信息传输用于视频压缩

Efficient Video Diffusion with Sparse Information Transmission for Video Compression

Shanghai Jiao Tong University(上海交通大学)

Mingde Zhou, Zheng Chen, Yulun Zhang

AI总结:

本文提出Diff-SIT方法,通过稀疏时序编码模块和单步视频扩散模型提升视频压缩的感知质量和时间一致性,尤其在超低比特率下表现优异。

摘要:

视频压缩旨在以最小比特率最大化重建质量。除了标准失真度量外,感知质量和时间一致性也至关重要。然而,在超低比特率下,传统端到端压缩模型常产生感知质量差的模糊图像。此外,现有生成压缩方法往往独立处理视频帧,存在时间一致性和效率的局限。为此,我们提出高效视频扩散与稀疏信息传输(Diff-SIT),包含稀疏时序编码模块(STEM)和单步视频扩散与帧类型嵌入器(ODFTE)。STEM将原始帧序列稀疏编码为信息丰富的中间序列,实现显著的比特率节省。随后,ODFTE处理此中间序列,利用时间相关性。在此过程中,所提的帧类型嵌入器(FTE)指导扩散模型根据不同帧类型进行自适应重建,以优化整体质量。在多个数据集上的广泛实验表明,Diff-SIT在感知质量和时间一致性方面建立了新的状态-of-the-art,特别是在挑战性的超低比特率环境下。代码已发布于https://github.com/MingdeZhou/Diff-SIT。

英文摘要:

Video compression aims to maximize reconstruction quality with minimal bitrates. Beyond standard distortion metrics, perceptual quality and temporal consistency are also critical. However, at ultra-low bitrates, traditional end-to-end compression models tend to produce blurry images of poor perceptual quality. Besides, existing generative compression methods often treat video frames independently and show limitations in time coherence and efficiency. To address these challenges, we propose the Efficient Video Diffusion with Sparse Information Transmission (Diff-SIT), which comprises the Sparse Temporal Encoding Module (STEM) and the One-Step Video Diffusion with Frame Type Embedder (ODFTE). The STEM sparsely encodes the original frame sequence into an information-rich intermediate sequence, achieving significant bitrate savings. Subsequently, the ODFTE processes this intermediate sequence as a whole, which exploits the temporal correlation. During this process, our proposed Frame Type Embedder (FTE) guides the diffusion model to perform adaptive reconstruction according to different frame types to optimize the overall quality. Extensive experiments on multiple datasets demonstrate that Diff-SIT establishes a new state-of-the-art in perceptual quality and temporal consistency, particularly in the challenging ultra-low-bitrate regime. Code is released at https://github.com/MingdeZhou/Diff-SIT.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑