arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2510.26254cs.CL

语言模型是否缺乏借词识别能力?对10种语言中借词识别的多语言评估

Are Language Models Borrowing-Blind? A Multilingual Evaluation of Loanword Identification across 10 Languages

Mérilin Sousa Silva, Sina Ahmadi

AI总结:

本文探讨了预训练语言模型在多语言环境下识别借词的能力,发现模型在区分借词与本土词汇方面表现不佳,对少数语言NLP工具开发和语言保护有重要意义。

摘要:

在语言历史中,词语会从一种语言借入另一种语言,并逐渐融入受体语言的词典中。说话者通常能够区分这些借词与本土词汇,特别是在双语社区中,当一种主导语言持续向少数语言施加词汇时更为明显。本文研究预训练语言模型,包括大语言模型,是否具备类似的借词识别能力。我们对10种语言的多个模型进行了评估。尽管有明确的指示和上下文信息,我们的结果表明,模型在区分借词与本土词汇方面表现不佳。这些发现支持了之前关于现代NLP系统倾向于偏爱借词而非本土词汇的证据。我们的工作对开发支持少数语言的NLP工具以及在受主导语言词汇压力的社区中支持语言保护具有意义。

英文摘要:

Throughout language history, words are borrowed from one language to another and gradually become integrated into the recipient's lexicon. Speakers can often differentiate these loanwords from native vocabulary, particularly in bilingual communities where a dominant language continuously imposes lexical items on a minority language. This paper investigates whether pretrained language models, including large language models, possess similar capabilities for loanword identification. We evaluate multiple models across 10 languages. Despite explicit instructions and contextual information, our results show that models perform poorly in distinguishing loanwords from native ones. These findings corroborate previous evidence that modern NLP systems exhibit a bias toward loanwords rather than native equivalents. Our work has implications for developing NLP tools for minority languages and supporting language preservation in communities under lexical pressure from dominant languages.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑