ResDiffFRG：用于多种适当面部反应生成的残差扩散

中文摘要 AI 辅助

在双人说话者-听者对话中，听者的面部反应使说话者能够准确感知听者的情绪状态。由于人类面部反应是非确定性的，生成多种适当的人性化面部反应的能力对于现实的人机交互至关重要。尽管扩散模型天然适合这种一对多生成，但现有的基于扩散的多种适当面部反应生成（MAFRG）方法试图直接将随机高斯初始化去噪为多种适当面部反应（AFR）。这些随机初始化通常与目标听者面部反应对齐不佳，这需要从这些初始化出发的复杂去噪轨迹，并随后为偏离通向适当AFR的轨迹范围创造了大量机会。鉴于人类听者与说话者面部行为之间的固有模仿性，我们通过利用这一强先验来解决上述去噪轨迹问题。具体来说，我们提出了ResDiffFRG，一种新颖的基于扩散的MAFRG框架，通过将其扩散目标定义为说话者锚点与AFR之间的残差，明确地将扩散过程锚定到说话者行为上。去噪器只需建模将这一锚点转换为AFR所需的相对较小的、反应特定的残差，而不是从非结构化状态重建完整反应。大量实验表明，ResDiffFRG在基于相关性的适当性方面比现有方法取得了大幅改进。我们的去噪轨迹分析表明，即使在去噪轨迹开始时，ResDiffFRG已经达到了比高斯扩散基线在完成其去噪轨迹60%后更高的面部反应相关性分数。

英文摘要

In dyadic human speaker-listener conversations, the listener's facial reactions allows the speaker to accurately perceive the listener's emotional states. Since human facial reactions are non-deterministic, the ability to generate multiple appropriate human-like facial reactions is crucial for realistic human-agent interactions. Although diffusion models are naturally suited to such one-to-many generation, existing diffusion-based Multiple Appropriate Facial Reaction Generation (MAFRG) methods attempt to denoise random Gaussian initialisations directly into multiple appropriate facial reactions (AFRs). These random initialisations are usually not well-aligned with the target listener facial reaction, which requires complex denoising trajectories from these initialisations, and subsequently creates substantial opportunities for deviations away from the range of trajectories leading to appropriate AFRs. Given the inherent mimicry between the human listener's and speaker's facial behaviours, we address the above denoising trajectory issue by leveraging this strong prior. Specifically, we propose ResDiffFRG, a novel diffusion-based MAFRG framework that explicitly anchors the diffusion process to the speaker behaviour by defining its diffusion target as the residual between the speaker anchor and an AFR. The denoiser only needs to model the comparatively small, reaction-specific residual needed to transform this anchor into an AFR, rather than reconstructing the complete reaction from an unstructured state. Extensive experiments show that ResDiffFRG achieves large improvements in correlation-based appropriateness over existing methods. Our denoising trajectory analysis showed that even at the start of the denoising trajectory, ResDiffFRG already achieves a higher facial-reaction correlation score than the Gaussian Diffusion baseline does after completing 60% of its denoising trajectory.

发表机构

• University of Oxford（牛津大学）
• Hefei University（合肥大学）
• University of Exeter（埃克塞特大学）

机构由 AI 辅助整理，请以论文原文为准。