CULEMO：情感的文化视角——跨文化情感理解的大语言模型基准测试
CULEMO: Cultural Lenses on Emotion -- Benchmarking LLMs for Cross-Cultural Emotion Understanding
AI总结：
针对现有情感基准忽略文化维度及翻译导致评估不可靠的问题，提出首个跨六种语言的文化感知情感预测基准CuLEmo，评估多个LLM并揭示了情感概念化与模型性能的跨文化差异及英语提示的优势。
摘要：
NLP研究越来越关注诸如情感分析等主观任务。然而，现有的情感基准存在两个主要缺陷：（1）它们主要依赖基于关键词的情感识别，忽略了更深层次情感理解所需的关键文化维度；（2）许多基准是通过将英语标注数据翻译成其他语言创建的，导致评估可能不可靠。为了解决这些问题，我们引入了Cultural Lenses on Emotion（CuLEmo），这是首个旨在评估跨六种语言（阿姆哈拉语、阿拉伯语、英语、德语、印地语和西班牙语）的文化感知情感预测的基准。CuLEmo包含每种语言400个精心设计的问题，每个问题都需要细致的文化推理与理解。我们使用该基准评估了多个最先进的LLM在文化感知情感预测和情感分析任务上的表现。我们的发现表明：（1）情感概念化在不同语言和文化间存在显著差异；（2）LLM的性能同样因语言和文化背景而异；（3）在文化感知情感与情绪理解任务中，带有明确国家背景的英语提示通常优于使用目标语言的提示。数据集和评估代码已公开提供。
英文摘要：
NLP research has increasingly focused on subjective tasks such as emotion analysis. However, existing emotion benchmarks suffer from two major shortcomings: (1) they largely rely on keyword-based emotion recognition, overlooking crucial cultural dimensions required for deeper emotion understanding, and (2) many are created by translating English-annotated data into other languages, leading to potentially unreliable evaluation. To address these issues, we introduce Cultural Lenses on Emotion (CuLEmo), the first benchmark designed to evaluate culture-aware emotion prediction across six languages: Amharic, Arabic, English, German, Hindi, and Spanish. CuLEmo comprises 400 crafted questions per language, each requiring nuanced cultural reasoning and understanding. We use this benchmark to evaluate several state-of-the-art LLMs on culture-aware emotion prediction and sentiment analysis tasks. Our findings reveal that (1) emotion conceptualizations vary significantly across languages and cultures, (2) LLMs performance likewise varies by language and cultural context, and (3) prompting in English with explicit country context often outperforms in-language prompts for culture-aware emotion and sentiment understanding. The dataset and evaluation code are publicly available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
