通过结构约束将无监督词对齐扩展至文档级别
Scaling Unsupervised Word Alignment to Documents via Structural Constraints
浏览论文内容
中文总结 AI 辅助
针对直接将句子级词对齐算法用于文档会导致性能下降的问题，提出CTFAlign和MDPAlign两种无需训练的文档级词对齐方法，经多语言对验证可降低词对齐错误率并提升下游任务表现。
中文摘要 AI 辅助
词对齐传统上在句子间进行研究，但许多跨语言任务日益需要完整文档间的对应关系。虽然近期多语言嵌入模型可编码长输入，但我们表明将针对句子设计的算法直接应用于文档会导致性能下降。为解决此问题，我们引入CTFAlign，这是一种轻量、无需训练的文档级词对齐方法。CTFAlign采用由粗到细的优化策略，将对齐搜索空间限制在语义相似的区域。此外，我们引入MDPAlign，这是一种更简单的替代方案，通过主对角线先验按位置约束对齐。两种方法均直接在完整文档上运行，无需依赖句子分割或句子对齐。我们在六种类型距离、资源丰富度和文档长度各异的语言对上评估这些方法。在三个模型上取平均，CTFAlign将词对齐错误率从0.412降至0.326。这些提升可传递至下游任务，带来文档级翻译覆盖度评估和语义差异识别的改进。我们将CTFAlign作为Python包发布，并公开了可复现实验的代码和数据。
英文摘要
Word alignment has traditionally been studied between sentences, but many cross-lingual tasks increasingly require correspondences across full documents. While recent multilingual embedding models can encode long inputs, we show that applying algorithms designed for sentences directly to documents leads to performance degradation. To address this, we introduce CTFAlign, a lightweight, training-free approach for document-level word alignment. CTFAlign applies a coarse-to-fine refinement strategy that restricts the alignment search space to semantically similar regions. Additionally, we introduce MDPAlign, a simpler alternative that constrains alignments by position with a main diagonal prior. Both approaches operate directly on full documents without relying on sentence segmentation or sentence alignment. We evaluate these methods across six language pairs varying in typological distance, resourcedness, and document length. Averaged over three models, CTFAlign reduces word alignment error rate from 0.412 to 0.326. These gains transfer downstream, leading to improvements in document-level translation coverage evaluation and recognition of semantic differences. We release CTFAlign as a Python package and make the code and data to reproduce our experiments publicly available.
发表机构
- University of Zurich（苏黎世大学）
机构由 AI 辅助整理，请以论文原文为准。
