FlowPortrait：基于强化学习的音频驱动肖像视频生成
FlowPortrait: Reinforcement Learning for Audio-Driven Portrait Video Generation
AI总结：
FlowPortrait通过强化学习框架结合多模态模型和人类对齐评估系统，提升音频驱动肖像视频生成的质量和表现力。
摘要：
生成逼真的说话人视频仍然具有挑战性，因为存在持续的问题，如不完美的唇同步、不自然的运动以及与人类感知相关的评估指标相关性差。我们提出了FlowPortrait，这是一种基于多模态骨干网络的强化学习框架，用于音频驱动的肖像动画生成。FlowPortrait引入了基于多模态大语言模型（MLLMs）的人类对齐评估系统，以评估唇同步准确性、表现力和运动质量。这些信号与感知和时间一致性正则化器相结合，形成稳定的复合奖励，用于通过分组相对策略优化（GRPO）后训练生成器。广泛的实验，包括自动评估和人类偏好研究，表明FlowPortrait一致地生成高质量的说话人视频，突显了强化学习在肖像动画中的有效性。
英文摘要：
Generating realistic talking-head videos remains challenging due to persistent issues such as imperfect lip synchronization, unnatural motion, and evaluation metrics that correlate poorly with human perception. We propose FlowPortrait, a reinforcement-learning framework for audio-driven portrait animation built on a multimodal backbone for autoregressive audio-to-video generation. FlowPortrait introduces a human-aligned evaluation system based on Multimodal Large Language Models (MLLMs) to assess lip-sync accuracy, expressiveness, and motion quality. These signals are combined with perceptual and temporal consistency regularizers to form a stable composite reward, which is used to post-train the generator via Group Relative Policy Optimization (GRPO). Extensive experiments, including both automatic evaluations and human preference studies, demonstrate that FlowPortrait consistently produces higher-quality talking-head videos, highlighting the effectiveness of reinforcement learning for portrait animation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
