arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.18429cs.CLcs.IR
收藏

VIRAASAT:为印度文化推理开辟新路径

VIRAASAT: Traversing Novel Paths for Indian Cultural Reasoning

AI Institute, University of South Carolina(AI研究所,南卡罗来纳大学);Indian Institute of Technology Patna(帕纳大学印度理工学院)

Harshul Raj Surana, Arijit Maji, Aryan Vats, Akash Ghosh, Sriparna Saha, Amit Sheth

AI总结:

VIRAASAT通过半自动化多跳方法生成印度文化多跳问题-答案数据集,提出SCoM框架提升文化推理性能。

摘要:

大型语言模型(LLMs)在数学和编码等各个领域的原因任务上取得了显著进展。然而,在需要丰富社会文化知识和多样本地背景的任务中,尤其是涉及印度文化的任务中,其性能会下降。现有的文化基准是(i)人工编写的,(ii)包含单跳问题测试事实记忆,(iii)扩展成本过高,导致这一缺陷大多未被测量。为了解决这个问题,我们引入了VIRAASAT,一种新颖的半自动化多跳方法,用于生成针对印度文化的多跳问题-答案数据集。VIRAASAT利用一个包含超过700个专家编写的文化物品的知识图谱,涵盖印度文化13个关键属性(历史、节日等)。VIRAASAT覆盖了所有28个州和8个联合领土,生成了超过3200个多跳问题,这些问题需要链式文化推理。我们评估了当前最先进的(SOTA)LLMs在VIRAASAT上的表现,并确定了推理中的关键限制,其中在链式思维(CoT)轨迹上微调未能定位和综合低概率事实。为了弥合这一差距,我们提出了一种名为符号链式操作(SCoM)的新框架。借鉴链式操作范式,我们训练模型内部模拟原子知识图谱操作。SCoM教会模型可靠地遍历图的拓扑结构。在监督微调(SFT)上的实验表明,SCoM比标准CoT基线高出多达20%。我们发布了VIRAASAT数据集以及我们的发现,为构建文化意识推理模型奠定了坚实基础。

英文摘要:

Large Language Models (LLMs) have made significant progress in reasoning tasks across various domains such as mathematics and coding. However, their performance deteriorates in tasks requiring rich socio-cultural knowledge and diverse local contexts, particularly those involving Indian Culture. Existing Cultural benchmarks are (i) Manually crafted, (ii) contain single-hop questions testing factual recall, and (iii) prohibitively costly to scale, leaving this deficiency largely unmeasured. To address this, we introduce VIRAASAT, a novel, semi-automated multi-hop approach for generating cultural specific multi-hop Question-Answering dataset for Indian culture. VIRAASAT leverages a Knowledge Graph comprising more than 700 expert-curated cultural artifacts, covering 13 key attributes of Indian culture (history, festivals, etc). VIRAASAT spans all 28 states and 8 Union Territories, yielding more than 3,200 multi-hop questions that necessitate chained cultural reasoning. We evaluate current State-of-the-Art (SOTA) LLMs on VIRAASAT and identify key limitations in reasoning wherein fine-tuning on Chain-of-Thought(CoT) traces fails to ground and synthesize low-probability facts. To bridge this gap, we propose a novel framework named Symbolic Chain-of-Manipulation (SCoM). Adapting the Chain-of-Manipulation paradigm, we train the model to simulate atomic Knowledge Graph manipulations internally. SCoM teaches the model to reliably traverse the topological structure of the graph. Experiments on Supervised Fine-Tuning (SFT) demonstrate that SCoM outperforms standard CoT baselines by up to 20%. We release the VIRAASAT dataset along with our findings, laying a strong foundation towards building Culturally Aware Reasoning Models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑