发表机构
The University of Sydney; Beijing University of Technology; Edith Cowan University（悉尼大学; 北京工业大学; 埃迪斯科文大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
针对多语言手语翻译中跨语言冲突问题，提出Q-BridgeNet框架，在手语侧用自适应分割等学习离散Q单元，口语侧微调多语言语言模型，实验证明该框架能有效减轻冲突，在多语言手语翻译上性能优异且泛化能力强。
AI 中文摘要
大多数手语翻译（SLT）方法专注于孤立的母语手语-口语对（如美国手语-英语）。将特定语言的SLT模型扩展到多语言翻译可促进不同手语和口语社区间交流。然而，现有多语言SLT方法难以学习统一模型以最小化跨语言冲突，同时捕捉共享跨语言语义并保留不同手语的特定语言变体。因此，我们提出Q-BridgeNet，一个多语言SLT统一框架，能减轻手语和口语两侧的跨语言冲突。在手语侧，通过自适应分割和残差向量量化学习离散Q单元；在口语侧，微调多语言语言模型在Q单元空间运行。实验表明Q-BridgeNet有效减轻跨语言冲突，在母语手语-口语对上取得最优性能，对非母语对也有强泛化能力。我们的源代码可公开获取。
英文摘要
Most sign language translation (SLT) methods focus on isolated native sign-spoken pairs (e.g., American Sign Language - English). Extending language-specific SLT models to multilingual translation would improve accessibility by enabling communication across diverse sign and spoken language communities. However, existing multilingual SLT approaches still struggle to learn a unified model that minimizes cross-lingual conflicts while capturing shared cross-lingual semantics and preserving language-specific variations across different sign languages. Therefore, we propose Q-BridgeNet, a unified framework for multilingual SLT that jointly mitigates cross-lingual conflicts across both the sign language and spoken language sides. On the sign language side, Q-BridgeNet learns discrete Q-units via adaptive segmentation and residual vector quantization: a shared base codebook provides language-agnostic semantic primitives, while language-specific residual codebooks refine heterogeneous signing semantics. On the spoken language side, a multilingual LLM is fine-tuned to operate in the Q-unit space, leveraging cross-lingual priors to enable a unified SLT model. Experiments on PHOENIX14T, How2Sign, and CSL-Daily show that Q-BridgeNet effectively mitigates cross-lingual conflicts, achieving state-of-the-art performance on native sign-spoken pairs while also demonstrating strong generalization to non-native pairs. Our source code is publicly available at: https://github.com/FengLiQ/Q-BridgeNet
