面向低码率说话人头部视频的音视频驱动压缩方法
Audio-Visual Driven Compression for Low-Bitrate Talking Head Videos
AI总结：
针对低码率说话人头部视频压缩的现有问题，提出融合紧凑3D运动特征与音频的音视频驱动编解码器，在CelebV-HQ数据集上实现码率降低与唇形同步、视觉质量的提升。
摘要：
说话人头部视频压缩已借助神经渲染和基于关键点的方法取得进展，但仍存在挑战，尤其是在低码率下，包括应对大幅头部运动、唇形同步效果不佳以及面部重建失真等问题。为解决这些问题，我们提出了一种新颖的音视频驱动视频编解码器，它融合了紧凑的3D运动特征与音频信号。该方法能鲁棒地对大幅头部旋转建模，并使唇部动作与语音对齐，同时提升压缩效率与重建质量。在CelebV-HQ数据集上的实验表明，我们的方法相比VVC码率降低22%，相比当前最优的基于学习的编解码器码率降低8.5%。此外，在相近码率下，该方法具备更优的唇形同步精度与视觉保真度，凸显了其在带宽受限场景下的有效性。
英文摘要：
Talking head video compression has advanced with neural rendering and keypoint-based methods, but challenges remain, especially at low bit rates, including handling large head movements, suboptimal lip synchronization, and distorted facial reconstructions. To address these problems, we propose a novel audio-visual driven video codec that integrates compact 3D motion features and audio signals. This approach robustly models significant head rotations and aligns lip movements with speech, improving both compression efficiency and reconstruction quality. Experiments on the CelebV-HQ dataset show that our method reduces bitrate by 22% compared to VVC and by 8.5% over state-of-the-art learning-based codec. Furthermore, it provides superior lip-sync accuracy and visual fidelity at comparable bitrates, highlighting its effectiveness in bandwidth-constrained scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
