arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.07461cs.CL
收藏

原生并行推理器:通过自我蒸馏强化学习实现并行推理

Native Parallel Reasoner: Reasoning in Parallelism via Self-Distilled Reinforcement Learning

State Key Laboratory of General Artificial Intelligence, BIGAI(通用人工智能国家重点实验室,BIGAI)

Tong Wu, Yang Liu, Jun Bai, Zixia Jia, Shuyi Zhang, Ziyong Lin, Yanting Wang, Song-Chun Zhu, Zilong Zheng

AI总结:

本文提出Native Parallel Reasoner,一种无需教师的框架,使大语言模型能够自我进化真正的并行推理能力。通过三个创新,NPR将模型从顺序模拟转变为原生并行认知,提升了推理性能和推理速度。

摘要:

我们介绍了Native Parallel Reasoner (NPR),一种无需教师的框架,使大语言模型 (LLMs) 能够自我进化真正的并行推理能力。NPR通过三个关键创新将模型从顺序模拟转变为原生并行认知:1) 一种自我蒸馏的渐进训练范式,从“冷启动”格式发现过渡到严格的拓扑约束,无需外部监督;2) 一种新的并行感知策略优化 (PAPO) 算法,直接在执行图中优化分支策略,使模型通过试错学习适应性分解;3) 一种稳健的NPR引擎,通过重构SGLang的内存管理和流控制,实现稳定的大规模并行强化学习训练。在八个推理基准测试中,NPR在Qwen3-4B上实现了高达24.5%的性能提升和高达4.6倍的推理加速。与以往基线方法通常回退到自回归解码不同,NPR展示了100%真正的并行执行,建立了自我进化、高效且可扩展的智能体推理的新标准。

英文摘要:

We introduce Native Parallel Reasoner (NPR), a teacher-free framework that enables Large Language Models (LLMs) to self-evolve genuine parallel reasoning capabilities. NPR transforms the model from sequential emulation to native parallel cognition through three key innovations: 1) a self-distilled progressive training paradigm that transitions from ``cold-start'' format discovery to strict topological constraints without external supervision; 2) a novel Parallel-Aware Policy Optimization (PAPO) algorithm that optimizes branching policies directly within the execution graph, allowing the model to learn adaptive decomposition via trial and error; and 3) a robust NPR Engine that refactors memory management and flow control of SGLang to enable stable, large-scale parallel RL training. Across eight reasoning benchmarks, NPR trained on Qwen3-4B achieves performance gains of up to 24.5% and inference speedups up to 4.6x. Unlike prior baselines that often fall back to autoregressive decoding, NPR demonstrates 100% genuine parallel execution, establishing a new standard for self-evolving, efficient, and scalable agentic reasoning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑