保留什么，遗忘什么：大语言模型和智能体中内存压缩的率失真视角
What to Keep, What to Forget: A Rate--Distortion View of Memory Compaction in LLMs and Agents
AI总结：
研究大语言模型和智能体内存压缩问题，提出基于率失真视角统一分类各层方法，构建七轴分类法并转移机制，还针对相关问题未被测量及缺乏统一基准的情况，转化为提案、实验、原则并梳理开放问题。
摘要：
大语言模型及其构建的智能体，在记忆方面花费的计算和内存份额不断增加，如缓存注意力键值、携带长提示、维护循环状态等。四个不同研究社区各自学会了对其进行压缩，包括逐出或量化KV缓存、修剪或提炼提示、限制架构状态以及整合智能体内存。我们认为这些是一个问题的实例，即关于在资源预算下，以何种保真度保留或丢弃上下文衍生信息以保持下游任务效用的率失真决策。我们用一个压缩目标和一个层无关下界使这个视角精确化，构建了一个七轴分类法来统一分类各层方法，并在从未连接的层之间转移机制。调查发现，各层决定保留内容的信号是注意力大小或近期性，且都会在查询未知时丢弃后续查询所需信息。同时，虽然对单轮长上下文的压缩有仔细测量，但智能体实际执行的重复压缩几乎未被测量，也没有一个基准能同时涵盖所有层的一个预算轴。我们将这些观察转化为一个基准提案、一个小型参考实验、一组压缩感知设计原则，并梳理了开放问题。
英文摘要：
Large language models, and the agents built on them, spend an ever-growing share of their compute and memory on remembering: caching attention keys and values, carrying long prompts, maintaining recurrent state, and storing what happened in previous turns and sessions. Because none of this memory is free, four largely separate research communities have each learned to compact it. They evict or quantize the KV cache, prune or distill prompts, bound architectural state, and consolidate agent memory. We argue that these are instances of one problem: a rate--distortion decision about what context-derived information to retain versus discard, at what fidelity, under a resource budget, so as to preserve downstream task utility. We make this lens precise with a single compaction objective and a layer-agnostic lower bound, use it to build a seven-axis taxonomy that classifies methods from across the stack uniformly, and use it to transfer mechanisms between layers that have never been connected, from serving-stack KV management to agent long-term memory. Two patterns hold across the survey. At every layer the signal that decides what to keep is attention magnitude or recency, and it fails in the same way everywhere, by discarding, before the query is known and with no way to undo it, information the query later needs. And while compression is measured carefully on single-turn long context, the repeated compaction that agents actually perform is almost never measured, and no benchmark holds one budget axis across all the layers at once. We turn both observations into a benchmark proposal, a small reference experiment, and a set of compaction-aware design principles, and we map the open problems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
