DISCO：通过接地-推理分离实现分布式长上下文扩展

中文摘要 AI 辅助

尽管大型语言模型（LLMs）宣称支持百万级token的上下文窗口，但随着输入增长，推理质量往往会崩溃——这一现象被称为上下文腐烂（context rot）。这种失败源于单体架构中的结构性纠缠，其中上下文接地的巨大搜索负担耗尽了复杂推理所需的表征能力。为解决此问题，我们提出了通过分布式长上下文扩展实现接地-推理分离（DISCO）。受Apache Spark等分布式计算框架启发，DISCO将长上下文分割到一组专门负责并行、局部接地的Worker LLM舰队中。一个中央Driver LLM，通过强化学习（GRPO）训练以优化规划，通过动态将查询映射为原子提取任务并整合收集的证据以综合最终答案来编排执行。通过将推理与原始上下文噪声隔离，DISCO有效消除了上下文腐烂。在RULER-QA（1M tokens）上，它保持了78.4%的准确率，而标准基线则崩溃。此外，它在LongBench v2上比全上下文模型高出最多9.8个百分点，并匹配了Gemini-3-Pro-Preview等前沿模型，同时将推理成本降低超过80%，为稳健的长上下文推理建立了一种高效范式。

英文摘要

While Large Language Models (LLMs) advertise million-token context windows, reasoning quality often collapses as inputs grow -- a phenomenon termed context rot. This failure stems from a structural entanglement in monolithic architectures, where the massive search burden of contextual grounding exhausts the representational capacity needed for complex reasoning. To resolve this, we propose Grounding-Reasoning Disaggregation via DIStributed long COntext scaling (DISCO). Inspired by distributed computing frameworks like Apache Spark, DISCO partitions long context across a fleet of Worker LLMs dedicated exclusively to parallel, localized grounding. A central Driver LLM, trained via Reinforcement Learning (GRPO) to optimize planning, orchestrates execution by dynamically mapping queries into atomic extraction tasks and reducing the gathered evidence to synthesize a final answer. By isolating reasoning from raw context noise, DISCO effectively eliminates context rot. On RULER-QA (1M tokens), it maintains 78.4% accuracy where standard baselines collapse. Furthermore, it outperforms full-context models by up to 9.8 points on LongBench v2 and matches frontier models like Gemini-3-Pro-Preview while reducing inference costs by over 80%, establishing a highly efficient paradigm for robust long-context inference.

发表机构

• National University of Singapore（新加坡国立大学）
• Adobe Research（奥多比研究院）

机构由 AI 辅助整理，请以论文原文为准。