arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.12881cs.CLcs.AI

BinaryAlign:将词对齐视为二元序列标注

BinaryAlign: Word Alignment as Binary Sequence Labeling

Ubisoft La Forge(育碧 La Forge)

Gaetan Lopez Latouche, Marc-André Carbonneau, Ben Swanson

AI总结:

BinaryAlign基于二元序列标注统一了高、低资源语言词对齐任务,在两种场景下均优于现有方法,并公开了源代码。

摘要:

词对齐的真实世界部署几乎必然覆盖高资源和低资源语言。然而,该任务的最先进方法根据特定语言对是否存在金标准对齐训练数据而推荐不同的模型类别。我们提出BinaryAlign,一种基于二元序列标注的新型词对齐技术,在两种场景下均优于现有方法,为该任务提供了统一的方法。此外,我们变化多语言基础模型的具体选择,按对齐错误类型进行分层错误分析,并探索BinaryAlign在非英语语言对上的性能。我们公开提供源代码。

英文摘要:

Real world deployments of word alignment are almost certain to cover both high and low resource languages. However, the state-of-the-art for this task recommends a different model class depending on the availability of gold alignment training data for a particular language pair. We propose BinaryAlign, a novel word alignment technique based on binary sequence labeling that outperforms existing approaches in both scenarios, offering a unifying approach to the task. Additionally, we vary the specific choice of multilingual foundation model, perform stratified error analysis over alignment error type, and explore the performance of BinaryAlign on non-English language pairs. We make our source code publicly available.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑