QwenLong-L1.5: 长上下文推理与内存管理的训练配方
QwenLong-L1.5: Post-Training Recipe for Long-Context Reasoning and Memory Management
AI总结:
QwenLong-L1.5通过长上下文数据合成、稳定强化学习和内存增强架构,提升长上下文推理能力及超长任务处理性能。
摘要:
我们介绍了QwenLong-L1.5,一种通过系统性训练创新实现卓越长上下文推理能力的模型。QwenLong-L1.5的关键技术突破包括:(1)长上下文数据合成流水线:我们开发了一个系统性合成框架,生成需要多跳支撑的复杂推理任务,通过将文档分解为原子事实及其底层关系,然后编程生成可验证的推理问题,从而生成高质量的训练数据,显著超越简单的检索任务,实现真正的长距离推理能力。(2)稳定强化学习用于长上下文训练:为克服长上下文强化学习中的关键不稳定性,我们引入任务平衡采样与任务特定优势估计以缓解奖励偏差,并提出自适应熵控制策略优化(AEPO)以动态调节探索-利用平衡。(3)内存增强架构用于超长上下文:认识到即使扩展的上下文窗口也无法容纳任意长的序列,我们开发了具有多阶段融合强化学习训练的内存管理框架,无缝整合单次推理与迭代内存处理,适用于超过400万token的任务。基于Qwen3-30B-A3B-Thinking,QwenLong-L1.5在长上下文推理基准测试中表现与GPT-5和Gemini-2.5-Pro相当,在平均上超越基线9.90点。在超长任务(100万~400万token)中,QwenLong-L1.5的内存代理框架相对于代理基线获得9.48点的提升。此外,获得的长上下文推理能力也转化为科学推理、内存工具使用和扩展对话等通用领域的增强性能。
英文摘要:
We introduce QwenLong-L1.5, a model that achieves superior long-context reasoning capabilities through systematic post-training innovations. The key technical breakthroughs of QwenLong-L1.5 are as follows: (1) Long-Context Data Synthesis Pipeline: We develop a systematic synthesis framework that generates challenging reasoning tasks requiring multi-hop grounding over globally distributed evidence. By deconstructing documents into atomic facts and their underlying relationships, and then programmatically composing verifiable reasoning questions, our approach creates high-quality training data at scale, moving substantially beyond simple retrieval tasks to enable genuine long-range reasoning capabilities. (2) Stabilized Reinforcement Learning for Long-Context Training: To overcome the critical instability in long-context RL, we introduce task-balanced sampling with task-specific advantage estimation to mitigate reward bias, and propose Adaptive Entropy-Controlled Policy Optimization (AEPO) that dynamically regulates exploration-exploitation trade-offs. (3) Memory-Augmented Architecture for Ultra-Long Contexts: Recognizing that even extended context windows cannot accommodate arbitrarily long sequences, we develop a memory management framework with multi-stage fusion RL training that seamlessly integrates single-pass reasoning with iterative memory-based processing for tasks exceeding 4M tokens. Based on Qwen3-30B-A3B-Thinking, QwenLong-L1.5 achieves performance comparable to GPT-5 and Gemini-2.5-Pro on long-context reasoning benchmarks, surpassing its baseline by 9.90 points on average. On ultra-long tasks (1M~4M tokens), QwenLong-L1.5's memory-agent framework yields a 9.48-point gain over the agent baseline. Additionally, the acquired long-context reasoning ability translates to enhanced performance in general domains like scientific reasoning, memory tool using, and extended dialogue.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。