超越 GFVC：一种基于自适应视觉 Token 的渐进式人脸视频压缩框架
Beyond GFVC: A Progressive Face Video Compression Framework with Adaptive Visual Tokens
AI总结：
本文提出渐进式人脸视频压缩框架 PFVC，利用自适应视觉 Token 实现重建鲁棒性与带宽智能的权衡，相比 VVC 和 GFVC 算法获得更优的编码灵活性和率失真性能。
摘要：
近年来，深度生成模型极大地推动了人脸视频编码朝着具有前景的率失真性能和多样化应用功能的方向发展。超越传统的混合视频编码范式，依赖深度生成模型强大能力与早期基于模型的编码（MBC）理念的生成式人脸视频压缩（GFVC），能够促进视觉人脸信号的紧凑表示和逼真重建，从而实现超低码率人脸视频通信。然而，这些 GFVC 算法有时面临重建质量不稳定和码率范围有限的问题。为了解决这些问题，本文提出了一种新颖的渐进式人脸视频压缩框架，即 PFVC，该框架利用自适应视觉 Token 来实现重建鲁棒性与带宽智能之间的出色权衡。具体而言，所提出的 PFVC 的编码器以渐进方式将高维人脸信号投影为自适应视觉 Token，而解码器则能进一步重建这些自适应视觉 Token，以进行不同粒度级别的运动估计和信号合成。实验结果表明，与最新的通用视频编码（VVC）编解码器和最先进的 GFVC 算法相比，所提出的 PFVC 框架能够实现更好的编码灵活性和更优的率失真性能。项目页面可在 https://github.com/Berlin0610/PFVC 找到。
英文摘要：
Recently, deep generative models have greatly advanced the progress of face video coding towards promising rate-distortion performance and diverse application functionalities. Beyond traditional hybrid video coding paradigms, Generative Face Video Compression (GFVC) relying on the strong capabilities of deep generative models and the philosophy of early Model-Based Coding (MBC) can facilitate the compact representation and realistic reconstruction of visual face signal, thus achieving ultra-low bitrate face video communication. However, these GFVC algorithms are sometimes faced with unstable reconstruction quality and limited bitrate ranges. To address these problems, this paper proposes a novel Progressive Face Video Compression framework, namely PFVC, that utilizes adaptive visual tokens to realize exceptional trade-offs between reconstruction robustness and bandwidth intelligence. In particular, the encoder of the proposed PFVC projects the high-dimensional face signal into adaptive visual tokens in a progressive manner, whilst the decoder can further reconstruct these adaptive visual tokens for motion estimation and signal synthesis with different granularity levels. Experimental results demonstrate that the proposed PFVC framework can achieve better coding flexibility and superior rate-distortion performance in comparison with the latest Versatile Video Coding (VVC) codec and the state-of-the-art GFVC algorithms. The project page can be found at https://github.com/Berlin0610/PFVC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
