缓解支付宝搜索中基于 LLM 的生成式检索幻觉
Alleviating LLM-based Generative Retrieval Hallucination in Alipay Search
AI总结：
本文提出融合知识蒸馏推理与决策智能体后处理的优化 GR 框架，以缓解支付宝搜索中基于 LLM 的生成式检索幻觉，并通过离线与在线实验验证其效果。
摘要：
随着大语言模型（LLM）的出现，生成式检索（GR）为文档检索带来了变革，基于 LLM 的 GR 也正逐渐被工业界采用。尽管其具有显著优势和潜力，但基于 LLM 的 GR 会受到幻觉影响，在某些情况下生成与查询无关的文档，这严重挑战了其在实际应用中的可信度。因此，我们提出了一个旨在缓解检索幻觉的优化 GR 框架，该框架在模型训练中融入知识蒸馏推理，并引入决策智能体以进一步提升检索精度。具体而言，我们使用 LLM 对 GR 检索到的查询-文档（q-d）对进行评估和推理，然后将推理数据作为迁移知识蒸馏到 GR 模型中。此外，我们利用决策智能体作为后处理，通过检索模型扩展 GR 检索到的文档，并从多个视角选择最相关的文档作为最终的生成式检索结果。在真实数据集上进行的大量离线实验，以及在支付宝基金搜索和保险搜索上开展的在线 A/B 测试，证明了我们的框架在提升搜索质量和转化收益方面的优越性与有效性。
英文摘要：
Generative retrieval (GR) has revolutionized document retrieval with the advent of large language models (LLMs), and LLM-based GR is gradually being adopted by the industry. Despite its remarkable advantages and potential, LLM-based GR suffers from hallucination and generates documents that are irrelevant to the query in some instances, severely challenging its credibility in practical applications. We thereby propose an optimized GR framework designed to alleviate retrieval hallucination, which integrates knowledge distillation reasoning in model training and incorporate decision agent to further improve retrieval precision. Specifically, we employ LLMs to assess and reason GR retrieved query-document (q-d) pairs, and then distill the reasoning data as transferred knowledge to the GR model. Moreover, we utilize a decision agent as post-processing to extend the GR retrieved documents through retrieval model and select the most relevant ones from multi perspectives as the final generative retrieval result. Extensive offline experiments on real-world datasets and online A/B tests on Fund Search and Insurance Search in Alipay demonstrate our framework's superiority and effectiveness in improving search quality and conversion gains.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
