SEA-NLI：将自然语言推理作为理解东南亚文化的透镜
SEA-NLI: Natural Language Inference as a Lens into Southeast Asian Cultural Understanding
AI总结：
提出SEA-NLI基准，通过自然语言推理评估模型对东南亚文化的理解，发现现有模型表现不佳，文化适应和提示可提升性能。
摘要：
前沿LLM在西方语境中表现良好，但在东南亚等代表性不足的文化中测试不足。现有的NLI基准大多以西方为中心、源自翻译或单语，限制了其衡量文化基础推理的能力。我们引入了SEA-NLI，一个原生的、基于文化的NLI基准，涵盖八个东南亚国家的英语和本地区域语言，并由母语者验证。在17个编码器和解码器模型中，我们观察到所有模型表现较低，尤其是在语言和科技等知识密集型类别中。我们的分析表明，失败案例主要源于缺乏东南亚文化知识：适应东南亚的模型和文化感知提示提升了性能，而思维链提示带来的提升有限。
英文摘要：
Frontier LLMs perform well in Western contexts, but remain poorly tested on underrepresented cultures such as those in Southeast Asia (SEA). Existing NLI benchmarks are largely Western-centric, translation-derived, or monolingual, limiting their ability to measure culturally grounded reasoning. We introduce SEA-NLI, a native, culturally grounded NLI benchmark covering eight SEA countries in English and native regional languages, verified by native speakers. Across 17 encoder and decoder models, we observe a low performance from all models, especially for knowledge-intensive categories such as Languages and Science and Technology. Our analysis shows that failure cases mainly stem from missing SEA cultural knowledge: SEA-adapted models and culture-aware prompting improve performance, while CoT prompting offers limited gains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
