LES-Talker：线性情感空间中面向说话头生成的细粒度情感编辑
LES-Talker: Fine-Grained Emotion Editing for Talking Head Generation in Linear Emotion Space
AI总结：
针对说话头生成中细粒度情感编辑可解释性不足的问题，本文提出 LES-Talker，通过基于面部动作单元的线性情感空间和跨维度注意力网络实现跨类型、跨强度、跨面部单元的细粒度可控情感编辑，并在视觉质量和编辑效果上优于主流方法。
摘要：
尽管现有的一次性说话头生成模型已在粗粒度情感编辑方面取得进展，但仍缺乏具有高可解释性的细粒度情感编辑模型。我们认为，一种方法若要被视为细粒度，就需要提供清晰的定义和足够细致的区分。我们提出 LES-Talker，一种具有高可解释性的新型一次性说话头生成模型，以实现跨情感类型、情感强度和面部单元的细粒度情感编辑。我们基于面部动作单元提出线性情感空间（LES）定义，将情感变换刻画为向量变换。我们设计跨维度注意力网络（CDAN），以深入挖掘 LES 表示与 3D 模型表示之间的相关性。通过挖掘不同特征维度和结构维度之间的多重关系，我们使 LES 表示能够引导 3D 模型的可控形变。为使存在偏差的多模态数据适配 LES 并提升视觉质量，我们采用专门的网络设计和训练策略。实验表明，我们的方法在提供高视觉质量的同时，实现了多层次且可解释的细粒度情感编辑，优于主流方法。
英文摘要：
While existing one-shot talking head generation models have achieved progress in coarse-grained emotion editing, there is still a lack of fine-grained emotion editing models with high interpretability. We argue that for an approach to be considered fine-grained, it needs to provide clear definitions and sufficiently detailed differentiation. We present LES-Talker, a novel one-shot talking head generation model with high interpretability, to achieve fine-grained emotion editing across emotion types, emotion levels, and facial units. We propose a Linear Emotion Space (LES) definition based on Facial Action Units to characterize emotion transformations as vector transformations. We design the Cross-Dimension Attention Net (CDAN) to deeply mine the correlation between LES representation and 3D model representation. Through mining multiple relationships across different feature and structure dimensions, we enable LES representation to guide the controllable deformation of 3D model. In order to adapt the multimodal data with deviations to the LES and enhance visual quality, we utilize specialized network design and training strategies. Experiments show that our method provides high visual quality along with multilevel and interpretable fine-grained emotion editing, outperforming mainstream methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
