DaKultur：以母语者评估语言模型对丹麦语的文化意识
DaKultur: Evaluating the Cultural Awareness of Language Models for Danish with Native Speakers
AI总结：
本文通过63名丹麦母语者对1,038次互动的评估，首次研究了LLM在丹麦语上的文化意识，发现自动翻译数据不足以衡量文化适应，而基于母语者数据训练可使回复接受率提升一倍以上，并发布了首个母语丹麦语文化意识数据集DaKultur。
摘要：
大型语言模型（LLMs）已获得广泛的社会应用。然而，尽管它们能够用英语以外的语言与用户互动，但已被证明缺乏文化意识，会为代表性不足的语言社区提供以盎格鲁为中心或不恰当的回复。为了调查这一差距并区分语言能力与文化能力，我们对中等资源的丹麦语进行了首次文化评估研究，其中母语者提示不同模型来解决需要文化意识的任务。我们对来自63名人口统计背景多样的参与者的1,038次互动的分析突显了文化适应的开放性挑战：特别是，当前使用的自动翻译数据不足以训练或衡量文化适应，以及在母语者数据上训练如何能使回复接受率提高一倍以上。我们将我们的研究数据作为DaKultur发布——这是首个母语丹麦语文化意识数据集。
英文摘要：
Large Language Models (LLMs) have seen widespread societal adoption. However, while they are able to interact with users in languages beyond English, they have been shown to lack cultural awareness, providing anglocentric or inappropriate responses for underrepresented language communities. To investigate this gap and disentangle linguistic versus cultural proficiency, we conduct the first cultural evaluation study for the mid-resource language of Danish, in which native speakers prompt different models to solve tasks requiring cultural awareness. Our analysis of the resulting 1,038 interactions from 63 demographically diverse participants highlights open challenges to cultural adaptation: Particularly, how currently employed automatically translated data are insufficient to train or measure cultural adaptation, and how training on native-speaker data can more than double response acceptance rates. We release our study data as DaKultur - the first native Danish cultural awareness dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
