arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.16852cs.CL
收藏

梅恩语仍为梅恩语,但大语言模型并不说它的方言

Meenz bleibt Meenz, but Large Language Models Do Not Speak Its Dialect

Johannes Gutenberg University Mainz, Germany(莱茵河畔摩泽尔大学);Marburg University, Germany(马堡大学);University of Colorado Boulder, USA(科罗拉多大学波德分校)

Minh Duc Bui, Manuel Mager, Peter Herbert Kann, Katharina von der Wense

AI总结:

本研究首次探讨梅恩语的NLP应用,发现大语言模型在生成方言词汇定义和生成方面表现不佳,需更多资源和研究支持。

摘要:

梅恩语是德国城市梅斯 spoken 的方言,也是梅斯狂欢节的传统语言,这是一种在德国广为人知的年度庆祝活动。然而,梅恩语正濒临灭绝——它与其他许多德国方言有着相同的命运。自然语言处理(NLP)有潜力帮助保护和复兴语言和方言。然而,迄今为止,还没有NLP研究关注梅恩语。本工作提出了在NLP领域首次专注于梅斯方言的研究。我们介绍了一个数字词典——一个从现有资源(Schramm, 1966)衍生出的NLP准备数据集,以支持研究人员在建模和基准测试语言方面的工作。它包含2,351个单词及其在标准德语中的意义描述。然后,我们使用这个数据集来回答以下研究问题:(1)最先进的大语言模型(LLMs)能否为方言单词生成定义?(2)LLMs能否根据定义生成梅恩语单词?我们的实验表明,LLMs无法做到这一点:生成定义的最佳模型仅达到6.27%的准确率,生成单词的最佳模型准确率为1.51%。然后,我们进行了两个额外的实验,以查看少量样本学习和从训练集提取规则是否能提高准确性,这些规则随后传递给LLM。虽然这些方法能够提高结果,但准确性仍低于10%。这突显了需要更多的资源和更加强调德国方言研究的科研努力。

英文摘要:

Meenzerisch, the dialect spoken in the German city of Mainz, is also the traditional language of the Mainz carnival, a yearly celebration well known throughout Germany. However, Meenzerisch is on the verge of dying out-a fate it shares with many other German dialects. Natural language processing (NLP) has the potential to help with the preservation and revival efforts of languages and dialects. However, so far no NLP research has looked at Meenzerisch. This work presents the first research in the field of NLP that is explicitly focused on the dialect of Mainz. We introduce a digital dictionary-an NLP-ready dataset derived from an existing resource (Schramm, 1966)-to support researchers in modeling and benchmarking the language. It contains 2,351 words in the dialect paired with their meanings described in Standard German. We then use this dataset to answer the following research questions: (1) Can state-of-the-art large language models (LLMs) generate definitions for dialect words? (2) Can LLMs generate words in Meenzerisch, given their definitions? Our experiments show that LLMs can do neither: the best model for definitions reaches only 6.27% accuracy and the best word generation model's accuracy is 1.51%. We then conduct two additional experiments in order to see if accuracy is improved by few-shot learning and by extracting rules from the training set, which are then passed to the LLM. While those approaches are able to improve the results, accuracy remains below 10%. This highlights that additional resources and an intensification of research efforts focused on German dialects are desperately needed.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑