超越艺术杰作或旅游陈词滥调：如何评估您的LLM文化对齐性？
Progressing beyond Art Masterpieces or Touristic Clichés: how to assess your LLMs for cultural alignment?
AI总结：
本文提出文化评估数据集的设计指南，并通过对比实验展示其在区分文化专精模型与非专精模型方面的有效性。
摘要：
尽管大型语言模型（LLM）的文化（不）对齐性问题日益受到关注——通常以文化偏见的角度来阐述——但直到最近，关于文化评估数据集设计和开发的工作仍然有限。本文回顾了现有数据集的方法并识别其主要局限性。为解决这些问题，我们提出了标注者的指导原则，并报告了根据这些原则构建的数据集。我们进一步使用此数据集进行了系列对比实验。结果表明，我们的设计产生了更具判别能力的测试集，有效地区分了专精于特定文化的模型和非专精模型。
英文摘要：
Although the cultural (mis)alignment of Large Language Models (LLMs) has attracted increasing attention -- often framed in terms of cultural bias -- until recently there has been limited work on the design and development of datasets for cultural assessment. Here, we review existing approaches to such datasets and identify their main limitations. To address these issues, we propose design guidelines for annotators and report on the construction of a dataset built according to these principles. We further present a series of contrastive experiments conducted with this dataset. The results demonstrate that our design yields test sets with greater discriminative power, effectively distinguishing between models specialized for a given culture and those that are not, ceteris paribus.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
