C2G2：基于潜在扩散模型的可控共语手势生成
C2G2: Controllable Co-speech Gesture Generation with Latent Diffusion Model
AI总结：
针对共语手势生成中训练不稳定和时间不一致的问题，提出C2G2框架，利用潜在扩散模型的两阶段时间依赖增强、说话者特定解码器和重绘策略，实现可控且可编辑的高保真手势生成，并在基准数据集上验证有效性。
摘要：
共语手势生成对于自动数字化身动画至关重要。然而，现有方法存在训练不稳定和时间不一致等问题，尤其是在生成高保真且全面的手势方面。此外，这些方法缺乏对说话者身份的有效控制以及对生成手势的时间编辑能力。为聚焦于捕捉时间潜在信息并应用实用控制，我们提出了一种名为C2G2的可控共语手势生成框架。具体而言，我们提出了一种受潜在扩散模型启发的两阶段时间依赖增强策略。我们进一步为C2G2引入了两个关键特性，即一个用于生成与说话者相关的真实长度骨骼的说话者特定解码器，以及一个用于灵活手势生成/编辑的重绘策略。在基准手势数据集上的大量实验验证了我们提出的C2G2相对于多个最先进基线的有效性。项目演示页面链接可在https://c2g2-gesture.github.io/c2_gesture找到。
英文摘要：
Co-speech gesture generation is crucial for automatic digital avatar animation. However, existing methods suffer from issues such as unstable training and temporal inconsistency, particularly in generating high-fidelity and comprehensive gestures. Additionally, these methods lack effective control over speaker identity and temporal editing of the generated gestures. Focusing on capturing temporal latent information and applying practical controlling, we propose a Controllable Co-speech Gesture Generation framework, named C2G2. Specifically, we propose a two-stage temporal dependency enhancement strategy motivated by latent diffusion models. We further introduce two key features to C2G2, namely a speaker-specific decoder to generate speaker-related real-length skeletons and a repainting strategy for flexible gesture generation/editing. Extensive experiments on benchmark gesture datasets verify the effectiveness of our proposed C2G2 compared with several state-of-the-art baselines. The link of the project demo page can be found at https://c2g2-gesture.github.io/c2_gesture
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
