X-NeMo：基于解耦潜在注意力的高表现力神经动作重现
X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention
AI总结：
本研究提出零样本扩散人像动画流水线X-NeMo，通过提取身份无关一维潜在动作描述符、交叉注意力控制及双GAN监督等设计，缓解身份泄露问题，提升表情表现力，效果优于现有最优基线。
摘要：
我们提出了X-NeMo，这是一种新颖的零样本、基于扩散的人像动画生成流水线，可利用来自不同个体的驱动视频中的面部动作让静态人像动起来。本研究首先明确了现有方法核心问题的根本原因，例如身份泄露以及难以捕捉细微和极端表情。为解决这些挑战，我们引入了一套完全端到端的训练框架，该框架从驱动图像中提取一维的身份无关潜在动作描述符，在图像生成过程中通过交叉注意力有效控制动作。我们的隐式动作描述符可精细捕捉富有表现力的面部动作，它从多样化的视频数据集中端到端学习得到，无需依赖预训练的动作检测器。我们进一步通过双GAN解码器监督动作潜在表示的学习，同时结合空间和颜色增强，提升了表现力，并将动作潜在表示与身份线索解耦。通过将驱动动作嵌入到一维潜在向量中，并通过交叉注意力而非加性空间引导来控制动作，我们的设计消除了空间对齐的结构线索从驱动条件向扩散主干的传递，大幅缓解了身份泄露问题。大量实验表明，X-NeMo 优于现有最优基线，能生成极具表现力的动画，且身份相似度更高。我们的代码和模型已开放供研究使用。
英文摘要：
We propose X-NeMo, a novel zero-shot diffusion-based portrait animation pipeline that animates a static portrait using facial movements from a driving video of a different individual. Our work first identifies the root causes of the key issues in prior approaches, such as identity leakage and difficulty in capturing subtle and extreme expressions. To address these challenges, we introduce a fully end-to-end training framework that distills a 1D identity-agnostic latent motion descriptor from driving image, effectively controlling motion through cross-attention during image generation. Our implicit motion descriptor captures expressive facial motion in fine detail, learned end-to-end from a diverse video dataset without reliance on pretrained motion detectors. We further enhance expressiveness and disentangle motion latents from identity cues by supervising their learning with a dual GAN decoder, alongside spatial and color augmentations. By embedding the driving motion into a 1D latent vector and controlling motion via cross-attention rather than additive spatial guidance, our design eliminates the transmission of spatial-aligned structural clues from the driving condition to the diffusion backbone, substantially mitigating identity leakage. Extensive experiments demonstrate that X-NeMo surpasses state-of-the-art baselines, producing highly expressive animations with superior identity resemblance. Our code and models are available for research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
