X-Portrait：基于分层运动注意力的表现力肖像动画
X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention
AI总结：
提出X-Portrait条件扩散模型，利用ControlNet框架和分层运动注意力机制，从单张肖像和驱动视频生成具有表现力且身份一致的肖像动画。
摘要：
我们提出了X-Portrait，一种专为生成具有表现力且时间连贯的肖像动画而设计的创新条件扩散模型。具体而言，给定单张肖像作为外观参考，我们的目标是利用从驱动视频中提取的运动对其进行动画化，捕捉高度动态和细微的面部表情以及大范围的头部运动。作为核心，我们利用预训练扩散模型的生成先验作为渲染骨干，同时在ControlNet框架内通过新颖的控制信号实现对头部姿态和表情的细粒度控制。与面部关键点等传统的粗粒度显式控制不同，我们的运动控制模块被训练为直接从原始驱动RGB输入中解读动态。通过基于补丁的局部控制模块，运动准确性得到进一步提升，该模块有效增强了对眼球位置等小尺度细节的运动注意力。值得注意的是，为了缓解驱动信号中的身份泄露问题，我们使用缩放增强的跨身份图像来训练运动控制模块，确保与外观参考模块实现最大程度的解耦。实验结果表明，X-Portrait在多种面部肖像和富有表现力的驱动序列上具有普遍的有效性，并展示了其在生成保持身份特征一致性的迷人肖像动画方面的卓越能力。
英文摘要：
We propose X-Portrait, an innovative conditional diffusion model tailored for generating expressive and temporally coherent portrait animation. Specifically, given a single portrait as appearance reference, we aim to animate it with motion derived from a driving video, capturing both highly dynamic and subtle facial expressions along with wide-range head movements. As its core, we leverage the generative prior of a pre-trained diffusion model as the rendering backbone, while achieve fine-grained head pose and expression control with novel controlling signals within the framework of ControlNet. In contrast to conventional coarse explicit controls such as facial landmarks, our motion control module is learned to interpret the dynamics directly from the original driving RGB inputs. The motion accuracy is further enhanced with a patch-based local control module that effectively enhance the motion attention to small-scale nuances like eyeball positions. Notably, to mitigate the identity leakage from the driving signals, we train our motion control modules with scaling-augmented cross-identity images, ensuring maximized disentanglement from the appearance reference modules. Experimental results demonstrate the universal effectiveness of X-Portrait across a diverse range of facial portraits and expressive driving sequences, and showcase its proficiency in generating captivating portrait animations with consistently maintained identity characteristics.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
