EpMAN:用于泛化到更长上下文的情景记忆注意力
EpMAN: Episodic Memory AttentioN for Generalizing to Longer Contexts
AI总结:
本文提出 EpMAN,通过情景记忆模块整体关注语义相关上下文块,并重加权解码器对 KV 缓存的自注意力,使 LLM 在 16k–256k tokens 的长上下文回忆与问答任务上优于自注意力基线和 RAG 框架。
摘要:
大语言模型(LLM)的最新进展已在许多语言任务上取得令人瞩目的成功。然而,使用 LLM 高效处理长上下文仍是一项重大挑战。我们提出 EpMAN——一种在情景记忆模块中处理长上下文的方法,同时整体关注语义相关的上下文块。随后,情景注意力的输出被用于在训练和生成期间,对解码器针对已存储上下文 KV 缓存的自注意力进行重新加权。当使用 EpMAN 训练 LLM 解码器时,在从 16k 到 256k tokens 的范围内,其在多项具有挑战性的单跳长上下文回忆和问答基准上的表现,比采用自注意力训练的基线解码器以及流行的检索增强生成框架更强、更稳健。
英文摘要:
Recent advances in Large Language Models (LLMs) have yielded impressive successes on many language tasks. However, efficient processing of long contexts using LLMs remains a significant challenge. We introduce \textbf{EpMAN} -- a method for processing long contexts in an \textit{episodic memory} module while \textit{holistically attending to} semantically relevant context chunks. The output of \textit{episodic attention} is then used to reweigh the decoder's self-attention to the stored KV cache of the context during training and generation. When an LLM decoder is trained using \textbf{EpMAN}, its performance on multiple challenging single-hop long-context recall and question-answering benchmarks is found to be stronger and more robust across the range from 16k to 256k tokens than baseline decoders trained with self-attention, and popular retrieval-augmented generation frameworks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。