通过自然辩论陈述理解多语言大语言模型中的文化契合
Understanding Cultural Alignment in Multilingual LLMs via Natural Debate Statements
AI总结:
本文通过分析自然辩论陈述,揭示了多语言大语言模型在不同文化背景下的价值观差异及其对用户社会文化背景适应能力的不足。
摘要:
在本工作中,我们研究了大型语言模型(LLMs)所学习的社会文化价值观。我们介绍了一个新的开放获取数据集,社会文化陈述,该数据集通过多步骤方法从自然辩论陈述中构建而成。该数据集经过合成标注,以量化LLMs在响应这些陈述时所表现出的社会文化规范和信念,依据霍夫斯泰德文化维度。我们通过人类质量控制对合成标签的准确性进行了验证,基于代表性样本。我们对两个不同国家(美国和中国)开发的LLM组进行了比较分析,并将人类测量中观察到的模式作为比较基准。使用这个新数据集和上述分析,我们发现文化不同的LLM反映了它们所开发国家的价值和规范,突显了它们无法适应其用户社会文化背景的不足。
英文摘要:
In this work we investigate the sociocultural values learned by large language models (LLMs). We introduce a novel open-access dataset, Sociocultural Statements, constructed from natural debate statements using a multi-step methodology. The dataset is synthetically labeled to enable the quantization of sociocultural norms and beliefs that LLMs exhibit in their responses to these statements, according to the Hofstede cultural dimensions. We verify the accuracy of synthetic labels using human quality control on a representative sample. We conduct a comparative analysis between two groups of LLMs developed in different countries (U.S. and China), and use as a comparative baseline patterns observed in human measurements. Using this new dataset and the analysis above, we found that culturally-distinct LLMs reflect the values and norms of the countries in which they were developed, highlighting their inability to adapt to the sociocultural backgrounds of their users.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。