Doc-to-LoRA: 学习即时内化上下文
Doc-to-LoRA: Learning to Instantly Internalize Contexts
AI总结：
Doc-to-LoRA通过轻量级超网络实现单次前向传递中的近似上下文蒸馏，提升大语言模型在长上下文任务中的推理效率和准确性。
摘要：
长输入序列是上下文学习、文档理解以及大语言模型（LLM）的多步推理的核心。然而，Transformer的二次注意力成本使得推理过程内存密集且缓慢。虽然上下文蒸馏（CD）可以将信息转移到模型参数中，但每提示蒸馏由于训练成本和延迟而不切实际。为了解决这些限制，我们提出了Doc-to-LoRA（D2L），一种轻量级的超网络，它元学习在单次前向传递中执行近似CD。给定一个未见过的提示，D2L为目标LLM生成一个LoRA适配器，使后续查询能够无需重新消耗原始上下文即可回答，从而减少目标LLM推理时的延迟和KV缓存内存消耗。在长上下文针在 haystack 任务中，D2L成功学习将上下文映射到适配器中存储针信息，实现序列长度超过目标LLM原生上下文窗口4倍以上时接近完美的零样本准确率。在计算资源有限的现实世界QA数据集上，D2L在显著降低峰值内存消耗和更新延迟的同时优于标准CD。我们设想D2L可以促进LLM的快速适应，开启频繁知识更新和个性化聊天行为的可能性。
英文摘要：
Long input sequences are central to in-context learning, document understanding, and multi-step reasoning of Large Language Models (LLMs). However, the quadratic attention cost of Transformers makes inference memory-intensive and slow. While context distillation (CD) can transfer information into model parameters, per-prompt distillation is impractical due to training costs and latency. To address these limitations, we propose Doc-to-LoRA (D2L), a lightweight hypernetwork that meta-learns to perform approximate CD within a single forward pass. Given an unseen prompt, D2L generates a LoRA adapter for a target LLM, enabling subsequent queries to be answered without re-consuming the original context, reducing latency and KV-cache memory consumption during inference of the target LLM. On a long-context needle-in-a-haystack task, D2L successfully learns to map contexts into adapters that store the needle information, achieving near-perfect zero-shot accuracy at sequence lengths exceeding the target LLM's native context window by more than 4x. On real-world QA datasets with limited compute, D2L outperforms standard CD while significantly reducing peak memory consumption and update latency. We envision that D2L can facilitate rapid adaptation of LLMs, opening up the possibility of frequent knowledge updates and personalized chat behavior.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
