基于轻量语义引导的扩散式极端视频压缩
Diffusion-aided Extreme Video Compression with Lightweight Semantics Guidance
AI总结：
本文提出一种基于扩散模型和轻量语义引导的视频压缩方法，通过整合生成先验知识和运动信息表征，实现极低比特率下的高质量视频重建。
摘要：
现代视频编解码器和基于学习的方法在极低比特率下难以实现语义重建，因为它们依赖于低级空间时间冗余性。生成模型，尤其是扩散模型，提供了一种新的视频压缩范式，通过利用高级语义理解和强大的视觉合成能力。本文提出了一种视频压缩框架，整合生成先验知识以大幅降低比特率同时保持重建保真度。具体而言，我们的方法压缩视频的高级语义表示，然后使用条件扩散模型从这些语义中重建帧。为进一步提高压缩效率，我们用全局相机轨迹和前景分割来表征运动信息：背景运动通过相机姿态参数紧凑表示，而前景动态通过稀疏分割掩码表示。这使得压缩效率显著提升，能够在极低比特率下实现高质量视频重建。
英文摘要：
Modern video codecs and learning-based approaches struggle for semantic reconstruction at extremely low bit-rates due to reliance on low-level spatiotemporal redundancies. Generative models, especially diffusion models, offer a new paradigm for video compression by leveraging high-level semantic understanding and powerful visual synthesis. This paper propose a video compression framework that integrates generative priors to drastically reduce bit-rate while maintaining reconstruction fidelity. Specifically, our method compresses high-level semantic representations of the video, then uses a conditional diffusion model to reconstruct frames from these semantics. To further improve compression, we characterize motion information with global camera trajectories and foreground segmentation: background motion is compactly represented by camera pose parameters while foreground dynamics by sparse segmentation masks. This allows for significantly boosts compression efficiency, enabling descent video reconstruction at extremely low bit-rates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
