基于StyleGAN的音频驱动高分辨率无缝说话人头部视频编辑
Audio-driven High-resolution Seamless Talking Head Video Editing via StyleGAN
AI总结：
针对现有音频驱动说话人头部视频编辑方法视觉效果差的问题，提出结合音频到面部关键点模块与基于StyleGAN的关键点编辑模块的方法，可生成高视觉质量的高分辨率无缝情绪编辑视频。
摘要：
现有音频驱动说话人头部视频编辑方法存在视觉效果不佳的局限性。本文尝试通过两个模块对不同情绪的说话人脸图像进行无缝编辑来解决该问题：（1）音频到 landmarks（面部关键点）模块，由交叉重构情绪解缠（CrossReconstructed Emotion Disentanglement）和对齐网络模块组成，通过从语音中预测对应的情绪关键点来弥合语音与面部动作之间的差距；（2）基于landmarks的编辑模块通过StyleGAN编辑人脸视频，旨在从输入音频中生成包含情绪与内容分量的无缝编辑视频。大量实验证实，与当前最优（state-of-the-arts）方法相比，本文方法能生成具有高视觉质量的高分辨率视频。
英文摘要：
The existing methods for audio-driven talking head video editing have the limitations of poor visual effects. This paper tries to tackle this problem through editing talking face images seamless with different emotions based on two modules: (1) an audio-to-landmark module, consisting of the CrossReconstructed Emotion Disentanglement and an alignment network module. It bridges the gap between speech and facial motions by predicting corresponding emotional landmarks from speech; (2) a landmark-based editing module edits face videos via StyleGAN. It aims to generate the seamless edited video consisting of the emotion and content components from the input audio. Extensive experiments confirm that compared with state-of-the-arts methods, our method provides high-resolution videos with high visual quality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
