arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.08382cs.CLcs.AI
收藏

通过端到端强化学习实现压缩内存中的动态长上下文推理

Dynamic Long Context Reasoning over Compressed Memory via End-to-End Reinforcement Learning

Zhuoen Chen, Dongfang Li, Meishan Zhang, Baotian Hu, Min Zhang

AI总结:

本文提出了一种基于端到端强化学习的框架,通过分块压缩和选择性记忆回溯实现高效长上下文推理,提升了多跳推理任务的准确性和效率。

摘要:

大型语言模型(LLMs)在长上下文处理中面临显著挑战,包括二次计算成本、信息遗忘以及检索增强生成(RAG)固有的上下文碎片化问题。我们提出了一种受认知启发的高效长上下文推理框架,该框架基于分块压缩和选择性记忆回溯,而非处理所有原始标记。该框架将长输入分成块,并使用学习得到的压缩器将每个块编码为压缩的内存表示。一个门控模块动态选择相关的内存块,这些块随后通过具有演进工作内存的推理模块迭代处理以解决下游任务。压缩器和推理器通过端到端强化学习联合优化,而门控模块作为分类器单独训练。实验结果表明,所提出的方法在多跳推理基准如RULER-HQA上实现了具有竞争力的准确性,能够将上下文长度从7K扩展到175万标记,并与强大的长上下文基线相比提供了有利的准确率-效率权衡。特别是,它在内存使用方面达到了峰值GPU内存使用量的两倍减少,并在MemAgent上实现了六倍的推理加速。

英文摘要:

Large Language Models (LLMs) face significant challenges in long-context processing, including quadratic computational costs, information forgetting, and the context fragmentation inherent in retrieval-augmented generation (RAG). We propose a cognitively inspired framework for efficient long-context inference based on chunk-wise compression and selective memory recall, rather than processing all raw tokens. The framework segments long inputs into chunks and encodes each chunk into compressed memory representations using a learned compressor. A gating module dynamically selects relevant memory blocks, which are then iteratively processed by a reasoning module with an evolving working memory to solve downstream tasks. The compressor and reasoner are jointly optimized via end-to-end reinforcement learning, while the gating module is trained separately as a classifier. Experimental results show that the proposed method achieves competitive accuracy on multi-hop reasoning benchmarks such as RULER-HQA, extrapolates context length from 7K to 1.75M tokens, and offers a favorable accuracy-efficiency trade-off compared to strong long-context baselines. In particular, it achieves up to a 2 times reduction in peak GPU memory usage and a 6 times inference speedup over MemAgent.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑