迈向语音引导的手语动作生成：一种扩散基线与条件分析
Toward Phonology-Guided Sign Language Motion Generation: A Diffusion Baseline and Conditioning Analysis
AI总结：
本文提出基于扩散模型的基线方法，研究文本条件对手语动作生成的影响，发现将符号化标注转为自然语言是CLIP有效条件，T5不受影响，最佳变体在所有指标上优于现有方法。
摘要：
生成自然、正确且视觉平滑的3D虚拟角色手语动作，条件于文本输入，仍极具挑战性。本文训练了一个3D身体动作生成模型，探索语音属性条件在手语动作生成中的作用，使用ASL-LEX 2.0标注如手形、手部位置和运动。我们首先建立了一个基于Human Motion MDM风格扩散模型的强基线，使用SMPL-X表示，优于最先进的CVAE方法SignAvatar在光泽可区分性度量上。然后系统研究了不同文本编码器（CLIP vs. T5）、条件模式（仅光泽 vs. 光泽+语音属性）和属性标注格式（符号 vs. 自然语言）的作用。我们的分析表明，将符号化ASL-LEX标注转换为自然语言是CLIP基于属性条件的有效必要条件，而T5则不受此转换影响。此外，我们的最佳变体（CLIP与映射属性）在所有指标上均优于SignAvatar。这些发现突显了输入表示作为文本编码器基于属性条件的关键因素，并激励了结构化条件方法，其中光泽和语音属性通过独立路径进行编码。
英文摘要：
Generating natural, correct, and visually smooth 3D avatar sign language motion conditioned on the text inputs continues to be very challenging. In this work, we train a generative model of 3D body motion and explore the role of phonological attribute conditioning for sign language motion generation, using ASL-LEX 2.0 annotations such as hand shape, hand location and movement. We first establish a strong diffusion baseline using an Human Motion MDM-style diffusion model with SMPL-X representation, which outperforms SignAvatar, a state-of-the-art CVAE method, on gloss discriminability metrics. We then systematically study the role of text conditioning using different text encoders (CLIP vs. T5), conditioning modes (gloss-only vs. gloss+phonological attributes), and attribute notation format (symbolic vs. natural language). Our analysis reveals that translating symbolic ASL-LEX notations to natural language is a necessary condition for effective CLIP-based attribute conditioning, while T5 is largely unaffected by this translation. Furthermore, our best-performing variant (CLIP with mapped attributes) outperforms SignAvatar across all metrics. These findings highlight input representation as a critical factor for text-encoder-based attribute conditioning, and motivate structured conditioning approaches where gloss and phonological attributes are encoded through independent pathways.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
