arXivDaily arXiv每日学术速递 周一至周五更新
​通知:联邦节假日调休:哥伦布日 / 原住民日(Columbus Day / Indigenous Peoples' Day),今日(13日)无更新。
arXiv 2410.02825cs.CLcs.CR

Ingest-And-Ground:用RAG消除持续预训练LLM的幻觉

Ingest-And-Ground: Dispelling Hallucinations from Continually-Pretrained LLMs with RAG

Meta

Chenhao Fang, Derek Larson, Shitong Zhu, Sophie Zeng, Wendy Summer, Yanqing Peng, Yuriy Hulovatyy, Rajeev Rao, Gabriel Forgues, Arya Pudota, Alex Goncalves, Hervé Robert

AI总结:

本文提出Ingest-And-Ground方法,通过隐私知识库持续预训练基础LLM并增强语义RAG层,有效减少幻觉,在隐私查询处理上性能指标较开箱即用LLM最多翻倍。

摘要:

本文提出有望利用LLM和RAG提升隐私流程效率的新方法。为减少幻觉,我们用隐私专属知识库持续预训练基础LLM模型,随后用语义RAG层对其进行增强。评估表明,该方法通过用事实信息为响应提供依据来减少不准确之处,从而在处理隐私相关查询时提升了模型性能(与开箱即用的LLM相比指标最多翻倍)。

英文摘要:

This paper presents new methods that have the potential to improve privacy process efficiency with LLM and RAG. To reduce hallucination, we continually pre-train the base LLM model with a privacy-specific knowledge base and then augment it with a semantic RAG layer. Our evaluations demonstrate that this approach enhances the model performance (as much as doubled metrics compared to out-of-box LLM) in handling privacy-related queries, by grounding responses with factual information which reduces inaccuracies.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑