基于代理思维链调优的长上下文推理
Long-Context Reasoning Through Proxy-Based Chain-of-Thought Tuning
AI总结：
提出ProxyCoT训练框架，通过将短代理上下文中的推理能力迁移到完整长上下文中，以提升大语言模型在长上下文复杂推理任务上的表现。
摘要：
近期的大语言模型支持高达1000万token的输入，但在需要复杂推理的长上下文任务上表现不佳。此类任务可以通过仅使用输入的一个子集（即代理上下文）而非完整序列来解决。尽管共享相同的底层推理过程，模型在代理上下文和完整上下文之间表现出显著的性能差异。为了改进长上下文推理，我们提出了ProxyCoT，一种新颖的训练框架，将推理能力从短代理上下文迁移到完整长上下文。具体来说，我们首先通过强化学习或从更大的教师模型蒸馏，在代理上下文中获得高质量的思维链推理轨迹，然后通过监督微调将这些生成的轨迹锚定到完整长上下文中。跨不同数据集的实验表明，ProxyCoT在减少计算开销的同时，始终优于强基线。此外，使用ProxyCoT训练的模型能够将其长上下文推理能力泛化到域外任务。
英文摘要：
Recent large language models support inputs of up to 10 million tokens, yet they perform poorly on long-context tasks that require complex reasoning. Such tasks can be solved using only a subset of the input -- a proxy context -- rather than the full sequence. Despite sharing the same underlying reasoning process, models exhibit a significant performance disparity between proxy and full contexts. To improve long-context reasoning, we propose ProxyCoT, a novel training framework that transfers reasoning capabilities from short proxy contexts to full long contexts. Specifically, we first obtain high-quality chain-of-thought reasoning traces on proxy contexts through reinforcement learning or distillation from a larger teacher model, and then ground the generated traces in full long contexts with supervised fine-tuning. Experiments across different datasets demonstrate that ProxyCoT consistently outperforms strong baselines with reduced computational overhead. Furthermore, models trained with ProxyCoT generalize their long-context reasoning capabilities to out-of-domain tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
