用于探索、提纯和模型合并的频谱重布线
Spectral Rewiring for Exploration, Purification, and Model Merging
AI总结：
研究强化学习中参数更新瓶颈，提出子空间对齐重布线方法，保留频谱核心去除正交分量，可提纯训练更新、实现模型合并，提升推理和跨域性能，是无训练机制提升性能。
摘要：
强化学习成大语言模型训练后标准方法，但密集全参数更新有瓶颈。研究表明更新中有效推理部分集中在频谱空间，提出子空间对齐重布线方法，保留频谱核心去除正交分量，能保留推理增益、过滤不良更新方向，提升多方面性能。
英文摘要：
Reinforcement learning has become a standard post-training recipe for large language models, but dense full-parameter updates create two deployment-relevant bottlenecks: suppressed reasoning performance, often reflected by premature saturation of test-time scaling, and interference when consolidating multiple capabilities through multi-domain training or model merging. We show that the reasoning-effective component of these updates is largely concentrated in the base model's spectral space, motivating Subspace-Aligned Rewiring (SAR), a post-hoc editing method that retains this spectral core while removing orthogonal components. SAR therefore preserves reasoning gains and filters residual update directions that suppress performance or amplify cross-domain interference. Across several model families and scales, SAR extracts compact reasoning cores using as little as approximately 0.58% of total parameters: it preserves over 99% of post-training performance and improves high-k exploration in mathematical reasoning, and generalizes to agentic coding by improving six of seven open benchmarks on an in-house model. SAR also purifies mixed-domain training updates by releasing suppressed coding capability while maintaining math reasoning and instruction following. It further enables model merging across experts, yielding cross-domain generalization that surpasses previous merging baselines and even the best single-domain experts. Overall, SAR shows that extracting reasoning-effective updates from parameter geometry can serve as a training-free mechanism to improve reasoning and multi-domain performance.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
