文化无处不在：呼吁有意识的文化评估
Culture is Everywhere: A Call for Intentionally Cultural Evaluation
AI总结：
本文批评现有LLM文化评估的琐事中心范式，提出有意识的文化评估，系统审视评估各环节的文化假设，强调研究者立场性及社区参与。
摘要：
随着大型语言模型（LLMs）变得愈发先进并被广泛部署，用于评估其文化对齐的主流“琐事中心范式”日益显得不足。现有方法通常将文化简化为静态事实或价值观，通过选择题或简答题来测试模型，将文化视为孤立的琐事。此类方法忽视了文化的多元性和互动性现实，也忽略了文化假设如何渗透进甚至表面上“中立”的评估设置中。在这篇立场论文中，我们主张进行“有意识的文化评估”：一种系统性地审视嵌入在评估各个方面（而不仅仅是明确的文化任务）中的文化假设的方法。我们系统地描述了文化偶然性考量在评估中出现的对象、方式及情境，并强调了研究者立场性对于促进包容性、文化对齐的NLP研究的重要性。最后，我们讨论了超越当前基准测试实践的影响和未来方向，发现我们尚不知道其存在的重要应用，并通过受HCI启发的参与式方法让社区参与评估设计。
英文摘要：
The prevailing ``trivia-centered paradigm'' for evaluating the cultural alignment of large language models (LLMs) is increasingly inadequate as these models become more advanced and widely deployed. Existing approaches typically reduce culture to static facts or values, testing models via multiple-choice or short-answer questions that treat culture as isolated trivia. Such methods neglect the pluralistic and interactive realities of culture, and overlook how cultural assumptions permeate even ostensibly ``neutral'' evaluation settings. In this position paper, we argue for \textbf{intentionally cultural evaluation}: an approach that systematically examines the cultural assumptions embedded in all aspects of evaluation, not just in explicitly cultural tasks. We systematically characterize the what, how, and circumstances by which culturally contingent considerations arise in evaluation, and emphasize the importance of researcher positionality for fostering inclusive, culturally aligned NLP research. Finally, we discuss implications and future directions for moving beyond current benchmarking practices, discovering important applications that we don't know exist, and involving communities in evaluation design through HCI-inspired participatory methodologies.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
