没有捷径的文化：印尼多跳问答用于复杂文化理解
No Shortcuts to Culture: Indonesian Multi-hop Question Answering for Complex Cultural Understanding
AI总结：
ID-MoCQA是一个基于印尼传统的多跳问答数据集，旨在评估大型语言模型的文化理解能力，揭示模型在文化推理方面的不足。
摘要：
理解文化需要在上下文、传统和隐含的社会知识之间进行推理，远超记忆孤立事实。然而，大多数专注于文化的问答（QA）基准测试依赖于单跳问题，这可能允许模型利用浅层线索，而不是展示真正的文化推理。在本工作中，我们引入了ID-MoCQA，这是首个大规模多跳问答数据集，用于评估大型语言模型（LLMs）的文化理解能力，基于印尼传统，并以英语和印尼语提供。我们提出了一种新的框架，系统地将单跳文化问题转换为跨越六个线索类型的多跳推理链（例如常识、时间、地理）。我们的多阶段验证流程，结合专家审查和LLM-as-a-judge过滤，确保高质量的问题-答案对。我们在最先进模型上的评估揭示了文化推理中的显著差距，特别是在需要细微推理的任务中。ID-MoCQA为推进LLMs的文化能力提供了具有挑战性和必要的基准。
英文摘要：
Understanding culture requires reasoning across context, tradition, and implicit social knowledge, far beyond recalling isolated facts. Yet most culturally focused question answering (QA) benchmarks rely on single-hop questions, which may allow models to exploit shallow cues rather than demonstrate genuine cultural reasoning. In this work, we introduce ID-MoCQA, the first large-scale multi-hop QA dataset for assessing the cultural understanding of large language models (LLMs), grounded in Indonesian traditions and available in both English and Indonesian. We present a new framework that systematically transforms single-hop cultural questions into multi-hop reasoning chains spanning six clue types (e.g., commonsense, temporal, geographical). Our multi-stage validation pipeline, combining expert review and LLM-as-a-judge filtering, ensures high-quality question-answer pairs. Our evaluation across state-of-the-art models reveals substantial gaps in cultural reasoning, particularly in tasks requiring nuanced inference. ID-MoCQA provides a challenging and essential benchmark for advancing the cultural competency of LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
