Pramana: 通过 Navya-Nyaya 微调大型语言模型进行认知推理
Pramana: Fine-Tuning Large Language Models for Epistemic Reasoning through Navya-Nyaya
AI总结：
提出 Pramana 方法，利用 2500 年历史的印度 Navya-Nyaya 逻辑框架微调 LLM，通过结构化六阶段推理解决认知差距，提升可溯源性。
摘要：
大型语言模型能生成流畅文本，但在系统推理方面存在困难，常常产生自信但无根据的幻觉。当苹果研究人员向数学问题添加无关背景时，LLM 性能下降了 65% (Apple Machine Learning Research)，暴露出表面推理下脆弱的模式匹配。这种认知差距，即无法将主张建立在可追溯证据上的能力，限制了 AI 在需要论证的领域的可靠性。我们引入 Pramana，一种新颖的方法，通过在 Navya-Nyaya 逻辑（一种 2500 年历史的印度推理框架）上进行微调，教导 LLM 显式的认识论方法论。与通用的思维链提示不同，Navya-Nyaya 强制执行结构化的六阶段推理：SAMSHAYA（怀疑分析）、PRAMANA（证据源识别）、PANCHA AVAYAVA（包含普遍规则的五段论）、TARKA（反事实验证）、HETVABHASA（谬误检测）和 NIRNAYA（区分知识与假设的确定）。这种逻辑与认识论的整合提供了标准推理方法所缺乏的认知支架。我们在 55 个 Nyaya 结构化的逻辑问题（约束满足、布尔 SAT、多步演绎）上微调了 Llama 3.2-3B 和 DeepSeek-R1-Distill-Llama-8B。第一阶段在保留评估上实现了 100% 的语义正确性，尽管严格格式遵循率仅为 40%，这表明即使结构执行不完美，模型也能内化推理内容。消融研究表明格式提示和温度对性能有关键影响，且不同阶段的最优配置不同。我们在 Hugging Face 上发布所有模型、数据集和训练基础设施，以促进关于 AI 推理认识论框架的进一步研究。
英文摘要：
Large language models produce fluent text but struggle with systematic reasoning, often hallucinating confident but unfounded claims. When Apple researchers added irrelevant context to mathematical problems, LLM performance degraded by 65% Apple Machine Learning Research, exposing brittle pattern-matching beneath apparent reasoning. This epistemic gap, the inability to ground claims in traceable evidence, limits AI reliability in domains requiring justification. We introduce Pramana, a novel approach that teaches LLMs explicit epistemological methodology by fine-tuning on Navya-Nyaya logic, a 2,500-year-old Indian reasoning framework. Unlike generic chain-of-thought prompting, Navya-Nyaya enforces structured 6-phase reasoning: SAMSHAYA (doubt analysis), PRAMANA (evidence source identification), PANCHA AVAYAVA (5-member syllogism with universal rules), TARKA (counterfactual verification), HETVABHASA (fallacy detection), and NIRNAYA (ascertainment distinguishing knowledge from hypothesis). This integration of logic and epistemology provides cognitive scaffolding absent from standard reasoning approaches. We fine-tune Llama 3.2-3B and DeepSeek-R1-Distill-Llama-8B on 55 Nyaya-structured logical problems (constraint satisfaction, Boolean SAT, multi-step deduction). Stage 1 achieves 100% semantic correctness on held-out evaluation despite only 40% strict format adherence revealing that models internalize reasoning content even when structural enforcement is imperfect. Ablation studies show format prompting and temperature critically affect performance, with optimal configurations differing by stage. We release all models, datasets, and training infrastructure on Hugging Face to enable further research on epistemic frameworks for AI reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
