SIGNER: 通过时间分辨条件实现时间对齐的手语生成
SIGNER: Temporally Grounded Sign Language Generation via Time-Resolved Conditioning
AI总结：
提出SIGNER框架，通过时间分辨条件（时间-词汇条件与局部时间融合）确保手语生成中词汇顺序正确和语义准确，在Phoenix-2014T和CSL-Daily上达到最优性能。
摘要：
手语生成（SLG），也称为文本到手语生成，旨在弥合手语使用者和非使用者之间的沟通鸿沟。与许多其他生成任务不同，SLG必须满足两个基本语言约束。首先，手语通过一系列与称为词汇（glosses）的词级单元对齐的手势来表达意义，因此需要正确的词汇顺序以保留预期意义。其次，每个手势应忠实反映预期的词汇（语义准确性）。尽管最近取得了进展，现有的SLG方法经常产生词汇顺序错误和语义准确性低的手语。先前方法的一个常见限制源于全局融合策略，这削弱了时间对齐——词汇与其实现的手势段之间的时间对应关系。这通常导致错误的词汇顺序和语义模糊的手势。为解决这一限制，我们提出SIGNER，一个具有时间分辨条件的手语生成框架，以确保时间对齐，利用时间-词汇条件和局部时间融合（LTF）。SIGNER通过从输入文本估计词汇序列及其持续时间，并在时间维度上分配词汇语义来构建时间-词汇条件。然后我们引入LTF，一个时间对齐的融合模块，在去噪过程中将时间-词汇条件整合到受限的时间窗口内。通过在条件融合中强制时间局部性，LTF保留了时间对齐，从而产生正确的词汇顺序和更清晰的每个词汇语义。在Phoenix-2014T和CSL-Daily上的实验展示了最先进的性能，并得到运动平滑度分析的进一步支持。项目页面可在此处获取：this https URL。
英文摘要：
Sign language generation (SLG), also known as text-to-sign generation, aims to bridge the communication gap between signers and non-signers. Unlike many other generative tasks, SLG must satisfy two fundamental linguistic constraints. First, sign language expresses meaning through a sequence of gestures aligned with word-like units called glosses, and therefore requires correct lexical ordering to preserve intended meaning. Second, each gesture should faithfully reflect the intended gloss (semantic accuracy). Despite recent progress, existing SLG methods frequently produce signs with incorrect lexical order and low semantic accuracy. A common limitation of prior approaches stems from globally fused conditioning strategies, which weaken temporal grounding, the temporal correspondence between glosses and their realized sign segments. This often leads to incorrect lexical order and semantically ambiguous signs. To address this limitation, we propose SIGNER, a SIGN language generation framework with timE-Resolved conditioning to ensure temporal grounding, leveraging a temporal-gloss condition and local temporal fusion (LTF). SIGNER constructs a temporal-gloss condition by estimating a gloss sequence and its durations from input text, and assigning gloss semantics across the temporal dimension. We then introduce LTF, a temporally grounded fusion module that integrates the temporal-gloss condition within a constrained temporal window during denoising. By enforcing temporal locality in condition fusion, LTF preserves temporal grounding, leading to correct lexical ordering and clearer per-gloss semantics. Experiments on Phoenix-2014T and CSL-Daily demonstrate state-of-the-art performance, further supported by motion-smoothness analysis. The project page is available here https://taeryunglee.github.io/projects/signer.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
