DeX-Portrait: 通过显式和潜在运动表示实现解耦且表达性的肖像动画
DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representations
AI总结：
DeX-Portrait通过显式和潜在运动表示生成解耦且表达性的肖像动画，提升动画质量和可控性。
摘要：
从单个源图像和驱动视频生成肖像动画是一个长期存在的问题。近期方法倾向于采用基于扩散的图像/视频生成模型来生成逼真且富有表现力的动画。然而，这些扩散模型均未能实现高保真度的头姿态与面部表情之间的解耦控制，阻碍了如仅表情或仅姿态编辑和动画等应用。为了解决这一问题，我们提出了DeX-Portrait，一种新的方法，能够通过解耦的姿态和表情信号生成富有表现力的肖像动画。具体来说，我们将姿态表示为显式的全局变换，表情表示为隐式的潜在代码。首先，我们设计了一个强大的运动训练器，用于学习姿态和表情编码器，以提取精确且分解的驱动信号。然后我们提出将姿态变换通过双分支条件机制注入扩散模型中，并通过交叉注意力机制注入表情潜在代码。最后，我们设计了一种渐进式的混合分类器自由引导机制，以实现更忠实的身份一致性。实验表明，我们的方法在动画质量和解耦可控性方面均优于最先进的基线方法。
英文摘要：
Portrait animation from a single source image and a driving video is a long-standing problem. Recent approaches tend to adopt diffusion-based image/video generation models for realistic and expressive animation. However, none of these diffusion models realizes high-fidelity disentangled control between the head pose and facial expression, hindering applications like expression-only or pose-only editing and animation. To address this, we propose DeX-Portrait, a novel approach capable of generating expressive portrait animation driven by disentangled pose and expression signals. Specifically, we represent the pose as an explicit global transformation and the expression as an implicit latent code. First, we design a powerful motion trainer to learn both pose and expression encoders for extracting precise and decomposed driving signals. Then we propose to inject the pose transformation into the diffusion model through a dual-branch conditioning mechanism, and the expression latent through cross attention. Finally, we design a progressive hybrid classifier-free guidance for more faithful identity consistency. Experiments show that our method outperforms state-of-the-art baselines on both animation quality and disentangled controllability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
