DiscoSign：篇章感知的文本到手语 gloss 翻译
DiscoSign: Discourse-Aware Text to Sign Language Gloss Translation
浏览论文内容
中文总结 AI 辅助
本研究提出基于LLM的DiscoSign框架，解决手语翻译中的空间共指等三个篇章现象，配套新评估指标，在数据集上显著提升空间一致性，建立首个篇章级手语gloss翻译系统框架
中文摘要 AI 辅助
手语处理系统传统上以句子级别运行，忽略了手语理解所必需的关键篇章现象。我们提出 DiscoSign，这是一种基于语言学研究的篇章感知文本到手语 gloss 翻译的计算方法。我们在模块化的基于大语言模型（LLM）的翻译框架中解决了三个关键现象：（i）空间共指消解，即实体在整个篇章中保持一致的空间位置；（ii）问答从句（QACs），即具有特定篇章功能的伪分裂结构；（iii）概念-gloss 一致性，确保英语概念与美国手语（ASL）符号之间的稳定映射。传统翻译指标无法捕捉篇章层面的质量，因此我们引入了一套新颖的评估指标，旨在评估我们框架所解决的篇章连贯性的每个维度。在句子级别和篇章级别数据集上的实验表明，我们的篇章感知处理方法与仅句子级别的翻译相比，显著提高了空间一致性和实体跟踪能力，同时保持了具有竞争力的单句 gloss 翻译质量。我们的工作建立了第一个用于篇章级文本到手语 gloss 翻译的系统框架及相应的评估方法。
英文摘要
Sign language processing systems have traditionally operated at the sentence level, ignoring critical discourse phenomena fundamental to sign language comprehension. We introduce DiscoSign, a computational approach for discourse-aware text to sign language gloss translation grounded in linguistic research. We address three key phenomena within our modular Large Language Model (LLM)-based translation framework: (i) spatial coreference resolution, where entities maintain consistent spatial locations throughout discourse; (ii) Question-Answer Clauses (QACs), pseudocleft structures serving specific discourse functions; and (iii) concept-gloss consistency, ensuring stable mappings between English concepts and American Sign Language (ASL) signs. Traditional translation metrics fail to capture discourse-level quality, so we introduce a suite of novel evaluation metrics designed to assess each dimension of discourse coherence addressed by our framework. Experiments on sentence-level and discourse-level datasets show that our approach for discourse-aware processing significantly improves spatial consistency and entity tracking relative to sentence-only translation, while maintaining competitive single-sentence gloss translation quality. Our work establishes the first systematic framework for discourse-level text to sign language gloss translation with corresponding evaluation methodology.
发表机构
- Apple（苹果公司）
- Northeastern University（东北大学）
- Gallaudet University（加劳德特大学）
机构由 AI 辅助整理，请以论文原文为准。
