DEMO：用于细粒度可控说话人肖像合成的解耦运动潜空间流匹配
DEMO: Disentangled Motion Latent Flow Matching for Fine-Grained Controllable Talking Portrait Synthesis
AI总结：
提出DEMO框架，通过运动自编码器构建解耦潜空间并结合流匹配生成模型，实现音频驱动说话人肖像视频中唇部、头部姿态和视线的高保真细粒度控制。
摘要：
基于音频驱动的说话人头像生成在基于扩散的生成模型推动下进展迅速，但生成具有细粒度运动控制的时间连贯视频仍具挑战。我们提出DEMO，一个用于音频驱动说话人肖像视频合成的流匹配生成框架，能对唇部运动、头部姿态和视线提供解耦且高保真的控制。核心贡献是一个运动自编码器，它构建了一个结构化潜空间，其中运动因子被独立表示并近似正交化。在此解耦运动空间上，我们应用基于最优传输的流匹配与transformer预测器，生成以音频为条件的平滑运动轨迹。跨多个基准的实验表明，DEMO在视频真实感、唇音同步和运动保真度上优于先前方法。结果表明，将细粒度运动解耦与基于流的生成建模结合，为可控说话人头像视频合成提供了强大新范式。
英文摘要：
Audio-driven talking-head generation has advanced rapidly with diffusion-based generative models, yet producing temporally coherent videos with fine-grained motion control remains challenging. We propose DEMO, a flow-matching generative framework for audio-driven talking-portrait video synthesis that delivers disentangled, high-fidelity control of lip motion, head pose, and eye gaze. The core contribution is a motion auto-encoder that builds a structured latent space in which motion factors are independently represented and approximately orthogonalized. On this disentangled motion space, we apply optimal-transport-based flow matching with a transformer predictor to generate temporally smooth motion trajectories conditioned on audio. Extensive experiments across multiple benchmarks show that DEMO outperforms prior methods in video realism, lip-audio synchronization, and motion fidelity. These results demonstrate that combining fine-grained motion disentanglement with flow-based generative modeling provides a powerful new paradigm for controllable talking-head video synthesis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
