RAIN-Merging: 一种无梯度方法，用于在保持推理格式的前提下提升大推理模型的指令遵循能力
RAIN-Merging: A Gradient-Free Method to Enhance Instruction Following in Large Reasoning Models with Preserved Thinking Format
AI总结：
RAIN-Merging通过无梯度方法提升大推理模型的指令遵循能力，同时保持推理格式和性能。
摘要：
大型推理模型（LRMs）在长链推理方面表现出色，但常常无法忠实遵循有关输出格式、约束或特定要求的指令。我们研究是否可以通过将指令调优模型（ITM）整合到LRM中来弥合这一差距。通过分析它们在参数空间中的差异，即任务向量，我们发现它们的主要子空间在关键模块中几乎正交，这表明可以进行轻量级的融合，干扰最小。然而，我们还证明了简单的融合是脆弱的，因为它们忽略了LRMs（具有显式推理和响应段）与ITMs（仅答案）之间的输出格式不匹配。我们引入了RAIN-Merging（基于推理的指令注意力引导的空域投影融合），一种无梯度方法，能够在保持推理格式和推理性能的同时整合指令遵循。首先，通过一个小的推理校准集，我们将ITM的任务向量投影到推理特殊标记的前向特征的空域中，这保留了LRM的结构化推理机制。其次，使用一个小的指令校准集，我们估计指令注意力以推导模块特定的缩放，放大与指令相关的组件并抑制泄漏。在四个指令遵循基准和九个推理与通用能力基准上，RAIN-Merging显著提高了指令遵循性，同时保持推理质量。这些收益在模型规模和架构上保持一致，转化为代理设置中的改进性能。
英文摘要：
Large reasoning models (LRMs) excel at a long chain of reasoning but often fail to faithfully follow instructions regarding output format, constraints, or specific requirements. We investigate whether this gap can be closed by integrating an instruction-tuned model (ITM) into an LRM. Analyzing their differences in parameter space, namely task vectors, we find that their principal subspaces are nearly orthogonal across key modules, suggesting a lightweight merging with minimal interference. However, we also demonstrate that naive merges are fragile because they overlook the output format mismatch between LRMs (with explicit thinking and response segments) and ITMs (answers-only). We introduce RAIN-Merging (Reasoning-Aware Instruction-attention guided Null-space projection Merging), a gradient-free method that integrates instruction following while preserving thinking format and reasoning performance. First, with a small reasoning calibration set, we project the ITM task vector onto the null space of forward features at thinking special tokens, which preserves the LRM's structured reasoning mechanisms. Second, using a small instruction calibration set, we estimate instruction attention to derive module-specific scaling that amplifies instruction-relevant components and suppresses leakage. Across four instruction-following benchmarks and nine reasoning & general capability benchmarks, RAIN-Merging substantially improves instruction adherence while maintaining reasoning quality. The gains are consistent across model scales and architectures, translating to improved performance in agent settings.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
