古老智慧,现代工具:探索检索增强LLM在古印度哲学中的应用
Ancient Wisdom, Modern Tools: Exploring Retrieval-Augmented LLMs for Ancient Indian Philosophy
AI总结:
针对LLM在专业领域易产生幻觉的问题,提出VedantaNY-10M数据集,开发并评估了RAG模型,证明其在古印度哲学长篇问答中显著优于标准模型,有效减少了幻觉。
摘要:
LLM彻底改变了信息检索和知识传播的格局。然而,它们在专业领域的应用常因事实不准确和幻觉而受阻,尤其是在长尾知识分布中。我们探索了检索增强生成(RAG)模型在专业知识领域长篇问答(LFQA)中的潜力。我们提出了VedantaNY-10M,一个从关于古印度哲学Advaita Vedanta的广泛公开演讲中整理的数据集。我们开发了一个RAG模型,并与标准的非RAG LLM进行了基准对比,重点关注转录、检索和生成性能。计算语言学家和领域专家的人工评估表明,RAG模型在产生事实准确、全面且回答幻觉更少方面显著优于标准模型。此外,强调独特低频词的基于关键词的混合检索器进一步改善了结果。我们的研究为将现代大型语言模型与古代知识系统有效整合提供了见解。项目页面(含数据集和代码):https://sites.google.com/view/vedantany-10m
英文摘要:
LLMs have revolutionized the landscape of information retrieval and knowledge dissemination. However, their application in specialized areas is often hindered by factual inaccuracies and hallucinations, especially in long-tail knowledge distributions. We explore the potential of retrieval-augmented generation (RAG) models for long-form question answering (LFQA) in a specialized knowledge domain. We present VedantaNY-10M, a dataset curated from extensive public discourses on the ancient Indian philosophy of Advaita Vedanta. We develop and benchmark a RAG model against a standard, non-RAG LLM, focusing on transcription, retrieval, and generation performance. Human evaluations by computational linguists and domain experts show that the RAG model significantly outperforms the standard model in producing factual and comprehensive responses having fewer hallucinations. In addition, a keyword-based hybrid retriever that emphasizes unique low-frequency terms further improves results. Our study provides insights into effectively integrating modern large language models with ancient knowledge systems. Project page with dataset and code: https://sites.google.com/view/vedantany-10m
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。