arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.16874cs.IRcs.AIcs.CL
收藏

通过增强查询提升语言生成的检索过程

Enhancing Retrieval Processes for Language Generation with Augmented Queries

Nagoya Institute of Technology(名古屋工业大学)

Julien Pierre Edmond Ghali, Kosuke Shima, Koichi Moriyama, Atsuko Mutoh, Nobuhiro Inuzuka

AI总结:

本研究针对大语言模型的幻觉问题,提出结合BERT、Orca2 7B与创新查询优化流程的RAG方案,通过三种场景实验验证其能显著提升检索生成性能,引入UMAP还可在保持效果的同时简化检索。

摘要:

在智能技术飞速发展的当下,由于先进语言模型的兴起,文档检索变得更具挑战性。这些模型有时会出现问题,比如提供不准确的信息,也就是人们常说的“幻觉”。本研究聚焦于通过检索增强生成(Retrieval-Augmented Generation,RAG)解决这一问题,RAG是一种引导模型基于真实事实给出准确响应的技术。为克服可扩展性问题,本研究探索将用户查询与BERT、Orca2等复杂语言模型相结合,采用一种创新的查询优化流程。研究在三种场景下展开:第一种不使用RAG,第二种无额外辅助,最后一种加入额外辅助。选择紧凑且高效的Orca2 7B模型体现了对计算资源的合理利用。实证结果表明,在RAG框架下,初始语言模型的性能得到显著提升,尤其是在提示增强器的辅助下。不同编码方式下文档检索结果的一致性凸显了使用语言模型生成查询的有效性。为BERT引入UMAP(均匀流形近似与投影)进一步简化了文档检索流程,同时保持了优异的结果。

英文摘要:

In the rapidly changing world of smart technology, searching for documents has become more challenging due to the rise of advanced language models. These models sometimes face difficulties, like providing inaccurate information, commonly known as "hallucination." This research focuses on addressing this issue through Retrieval-Augmented Generation (RAG), a technique that guides models to give accurate responses based on real facts. To overcome scalability issues, the study explores connecting user queries with sophisticated language models such as BERT and Orca2, using an innovative query optimization process. The study unfolds in three scenarios: first, without RAG, second, without additional assistance, and finally, with extra help. Choosing the compact yet efficient Orca2 7B model demonstrates a smart use of computing resources. The empirical results indicate a significant improvement in the initial language model's performance under RAG, particularly when assisted with prompts augmenters. Consistency in document retrieval across different encodings highlights the effectiveness of using language model-generated queries. The introduction of UMAP for BERT further simplifies document retrieval while maintaining strong results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑