BengaliFig：一种低资源挑战，用于孟加拉语中的隐喻和文化相关推理
BengaliFig: A Low-Resource Challenge for Figurative and Culturally Grounded Reasoning in Bengali
AI总结：
BengaliFig通过孟加拉语隐喻和文化相关推理的低资源挑战，评估LLM在文化特定推理中的表现，并推动包容性NLP评估。
摘要：
大型语言模型在广泛的多语言基准测试中表现优异，但在隐喻和文化相关推理方面仍需广泛评估，尤其是在低资源环境下。我们提出了BengaliFig，一个紧凑但注释丰富的挑战集，旨在填补孟加拉语中这一空白，孟加拉语是一种广泛使用的低资源语言。该数据集包含435个独特的谜语，来源于孟加拉语的口头和文学传统。每个项目都沿着五个正交维度进行注释，捕捉推理类型、陷阱类型、文化深度、答案类别和难度，并通过一个受约束的、由AI辅助的管道自动转换为多项选择格式。我们在零样本和少量样本的链式思维提示下评估了八个前沿LLM，揭示了在隐喻和文化特定推理方面的持续弱点。BengaliFig因此既为评估LLM在低资源文化环境中的鲁棒性提供了诊断探针，也朝着包容性和遗产意识的NLP评估迈进了一步。
英文摘要：
Large language models excel on broad multilingual benchmarks but remain to be evaluated extensively in figurative and culturally grounded reasoning, especially in low-resource contexts. We present BengaliFig, a compact yet richly annotated challenge set that targets this gap in Bengali, a widely spoken low-resourced language. The dataset contains 435 unique riddles drawn from Bengali oral and literary traditions. Each item is annotated along five orthogonal dimensions capturing reasoning type, trap type, cultural depth, answer category, and difficulty, and is automatically converted to multiple-choice format through a constraint-aware, AI-assisted pipeline. We evaluate eight frontier LLMs from major providers under zero-shot and few-shot chain-of-thought prompting, revealing consistent weaknesses in metaphorical and culturally specific reasoning. BengaliFig thus contributes both a diagnostic probe for evaluating LLM robustness in low-resource cultural contexts and a step toward inclusive and heritage-aware NLP evaluation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
