文化感知对话：面向LLM的框架与基准
Culturally-Aware Conversations: A Framework & Benchmark for LLMs
AI总结：
针对现有基准与LLM跨文化交互实际挑战错位的问题，提出基于社会文化理论的文化感知对话评估框架与基准，并引入对话框架等新评估要求，实验表明顶级LLM在对话场景中仍难以实现文化适应。
摘要：
现有衡量LLM文化适应性的基准与这些模型在跨文化用户交互中面临的实际挑战存在错位。本文提出首个用于在真实多元文化对话场景中评估LLM的框架与基准。基于社会文化理论，该框架形式化了语言风格——文化沟通的关键要素——如何受情境、关系和文化语境塑造。我们基于此框架构建基准数据集，由多元文化背景的标注者进行标注，并提出NLP跨文化评估的新要求集：对话框架、风格敏感性和主观正确性。我们在该基准上评估当今顶级LLM，结果表明这些模型在对话场景中难以进行文化适应。
英文摘要：
Existing benchmarks that measure cultural adaptation in LLMs are misaligned with the actual challenges these models face when interacting with users from diverse cultural backgrounds. In this work, we introduce the first framework and benchmark designed to evaluate LLMs in realistic, multicultural conversational settings. Grounded in sociocultural theory, our framework formalizes how linguistic style - a key element of cultural communication - is shaped by situational, relational, and cultural context. We construct a benchmark dataset based on this framework, annotated by culturally diverse raters, and propose a new set of desiderata for cross-cultural evaluation in NLP: conversational framing, stylistic sensitivity, and subjective correctness. We evaluate today's top LLMs on our benchmark and show that these models struggle with cultural adaptation in a conversational setting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
