arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2604.17718cs.CLcs.SI

大型语言模型是否在未被告知的情况下使用文化知识?一种多语言隐含语用适应性评估

Do LLMs Use Cultural Knowledge Without Being Told? A Multilingual Evaluation of Implicit Pragmatic Adaptation

Network Analysis and Social Influence Modelling (NASIM) Lab(网络分析与社会影响力建模实验室);School of Physics Maths and Computing(物理数学与计算学院);The University of Western Australia(西澳大学);School of Computing(计算学院);Macquarie University(麦考瑞大学);Computational Social Science & Systems Analysis(计算社会科学与系统分析);University of Münster(明斯特大学)

Mehwish Nasim, Sanjeevan Selvaganapathy, Neel Ganapathi Sabhahit, Marie Griesbach, Pranav Bhandari, Janina Lütke Stockdiek, Lennart Schäpermeier, Usman Naseem, Christian Grimme

AI总结:

本文研究了大型语言模型在隐含文化提示下是否改变其说话方式,通过五种语言的60个文化相关对话场景,评估了模型在不同提示条件下的语用特征,发现模型在隐含提示下只能部分恢复显式指令下的语用变化。

摘要:

许多基准测试显示,大型语言模型能够回答关于文化直接问题。我们研究了一个不同的问题:当文化仅通过情境暗示时,它们是否会改变说话方式?我们评估了五种语言中60个文化相关的对话场景,三种条件下:中性基线(提示A)、显式文化指令(提示B)和隐含情境提示(提示C)。我们对响应在12个语用特征上进行评分,涵盖对权威的敬重、个人与群体框架以及不确定性管理。我们定义语用情境敏感性(PCS)为提示A→B转变在提示A→C下再现的分数。在四个部署的LLM和五种语言(英语、德语、印地语、尼泊尔语、乌尔都语)中,主要的稳定仅PCS均值为0.196(SD=0.113),表明模型只能恢复大约五分之一的显式指令下能产生的语用变化。转移最强的是与权威相关的提示(0.299),最弱的是个人与群体框架(0.120)。与不确定性相关的行为混合:所有五种语言的让步密度都表现出负的显式差距,表明对齐训练积极抑制了目标行为。由于印地语和乌尔都语共享核心语法却指代不同的文化社区,我们使用它们作为自然对照;配对分析发现没有可靠的基线差异(t=0.96,p=0.339,dz=0.06),表明模型主要响应于语言结构而非语言所携带的文化关联。我们主张多语言文化语用学是显式与隐式部署问题,而不仅仅是事实知识问题。

英文摘要:

Many benchmarks show that large language models can answer direct questions about culture. We study a different question: do they also change how they speak when culture is only implied by the situation? We evaluate 60 culturally grounded conversational scenarios across five languages in three conditions: a neutral baseline (Prompt A), an explicit cultural instruction (Prompt B), and implicit situational cueing (Prompt C). We score responses on 12 pragmatic features covering deference to authority, individual-versus-group framing, and uncertainty management. We define Pragmatic Context Sensitivity (PCS) as the fraction of the Prompt A->B shift that reappears under Prompt A->C. Across four deployed LLMs and five languages (English, German, Hindi, Nepali, Urdu), the primary stable-only PCS mean is 0.196 (SD = 0.113), indicating that the models recover only about one-fifth of the pragmatic shift they can produce when instructed explicitly. Transfer is strongest for authority-related cues (0.299) and weakest for individual-versus-group framing (0.120). Uncertainty-related behaviour is mixed: hedging density exhibits negative explicit gaps in all five languages, suggesting that alignment training actively suppresses the target behaviour. Because Hindi and Urdu share core grammar yet index distinct cultural communities, we use them as a natural control; a paired analysis finds no reliable baseline difference (t = 0.96, p = 0.339, dz = 0.06), suggesting that models respond primarily to linguistic structure rather than to the cultural associations a language carries. We argue that multilingual cultural pragmatics is an explicit-versus-implicit deployment problem, not only a factual knowledge problem.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑