CRaFT：一种基于解释的多语言语言模型文化推理评估框架
CRaFT: An Explanation-Based Framework for Evaluating Cultural Reasoning in Multilingual Language Models
AI总结：
CRaFT是一个基于解释的多语言评估框架，通过文化流畅度等四个指标评估LLM跨文化推理，发现文化意识依赖语言框架而非固有。
摘要：
正确答案并不一定反映文化理解。我们提出了CRaFT，一个基于解释的多语言评估框架，旨在评估大型语言模型（LLM）在不同文化背景下的推理能力。CRaFT并非仅根据准确性对输出进行评分，而是使用四个可解释的指标来评估模型的解释：文化流畅度、偏离度、一致性和语言适应性。我们将该框架应用于来自世界价值观调查的50个具有文化根基的问题，这些问题被翻译成阿拉伯语、孟加拉语和西班牙语，并评估了三个模型（GPT、DeepSeek和FANAR）的超过2,100对答案-解释对。结果显示推理存在显著的跨语言差异：阿拉伯语降低了流畅度，孟加拉语提升了流畅度，而西班牙语基本保持稳定。虽然GPT在跨语言适应方面更有效，但其一致性较低；FANAR表现出稳定但僵化的推理。这些发现表明，LLM中的文化意识并非固有，而是通过语言框架浮现。CRaFT为评估多语言环境中的跨文化推理提供了新视角，为构建文化适应性语言模型提供了可操作的见解。
英文摘要：
Correct answers do not necessarily reflect cultural understanding. We introduce CRaFT, an explanation-based multilingual evaluation framework designed to assess how large language models (LLMs) reason across cultural contexts. Rather than scoring outputs solely based on accuracy, CRaFT evaluates model explanations using four interpretable metrics: Cultural Fluency, Deviation, Consistency, and Linguistic Adaptation. We apply the framework to 50 culturally grounded questions from the World Values Survey, translated into Arabic, Bengali, and Spanish, and evaluate three models (GPT, DeepSeek, and FANAR) across over 2,100 answer-explanation pairs. Results reveal significant cross-lingual variation in reasoning: Arabic reduces fluency, Bengali enhances it, and Spanish remains largely stable. While GPT adapts more effectively across languages, it exhibits lower consistency; FANAR shows stable but rigid reasoning. These findings suggest that cultural awareness in LLMs is not intrinsic but emerges through linguistic framing. CRaFT offers a new lens for evaluating cross-cultural reasoning in multilingual settings, providing actionable insights for building culturally adaptive language models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
