arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.13682cs.CLcs.AIcs.LG
收藏

通过Token消除优化检索增强阅读器模型

Optimizing Retrieval-augmented Reader Models via Token Elimination

Intel Labs, Israel(英特尔实验室(以色列));Bar-Ilan University, Israel(巴伊兰大学(以色列));Google Research(谷歌研究院)

Moshe Berchansky, Peter Izsak, Avi Caciularu, Ido Dagan, Moshe Wasserblat

AI总结:

针对检索增强模型FiD解码速度慢的问题,提出在token层面消除无用检索信息的优化方法,可大幅降低运行时间且性能损失极小,部分场景还能提升效果。

摘要:

Fusion-in-Decoder(FiD,解码器内融合)是一种有效的检索增强语言模型,应用于问答、事实验证等多种开放域任务。在FiD中,首先检索支撑段落,再用生成模型(阅读器)处理,这会在解码时间上造成显著瓶颈,长输出时尤为明显。本研究分析了所有检索段落对阅读器模型性能的贡献与必要性,提出在token层面消除部分对答案生成过程无关键信息的检索内容。实验表明,该方法可将运行时间最多降低62.2%,性能仅下降2%,部分场景下甚至能提升性能表现。

英文摘要:

Fusion-in-Decoder (FiD) is an effective retrieval-augmented language model applied across a variety of open-domain tasks, such as question answering, fact checking, etc. In FiD, supporting passages are first retrieved and then processed using a generative model (Reader), which can cause a significant bottleneck in decoding time, particularly with long outputs. In this work, we analyze the contribution and necessity of all the retrieved passages to the performance of reader models, and propose eliminating some of the retrieved information, at the token level, that might not contribute essential information to the answer generation process. We demonstrate that our method can reduce run-time by up to 62.2%, with only a 2% reduction in performance, and in some cases, even improve the performance results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑