思维谱系：通过模型合并实现大语言模型可调推理能力的实证研究
The Thinking Spectrum: An Empirical Study of Tunable Reasoning in LLMs through Model Merging
AI总结：
本文通过大规模实证研究系统改变模型合并强度，证明无需训练即可在推理准确率与 token 效率间实现可控权衡，并发现帕累托改进。
摘要：
在许多现实应用中，对具备可调推理能力的大语言模型（LLM）的需求日益增长，这凸显出一个关键需求：需要能够高效生成一系列模型、以平衡推理深度与计算成本的方法。模型合并已成为应对这一挑战的一种有前景且无需训练的技术，它通过算术方式将通用模型与专用推理模型的权重相结合。尽管已存在多种合并技术，但它们在创建一系列可对推理能力进行细粒度控制的模型方面的潜力，在很大程度上仍未得到探索。本工作开展了一项大规模实证研究，在多个推理基准上评估了一系列模型合并技术。我们系统地改变合并强度以构建准确率—效率曲线，首次全面呈现了可调性能图景。研究结果表明，即使父模型具有高度分化的权重空间，模型合并仍提供了一种有效且可控的方法，用于校准推理准确率与 token 效率之间的权衡。关键在于，我们发现了 Pareto Improvement（帕累托改进）情形：合并后的模型相比其某个父模型同时实现了更高的准确率和更低的 token 消耗。本研究首次对这一可调空间进行了全面分析，为创建具有特定推理配置的 LLM 以满足多样化应用需求提供了实用指南。
英文摘要：
The growing demand for large language models (LLMs) with tunable reasoning capabilities in many real-world applications highlights a critical need for methods that can efficiently produce a spectrum of models balancing reasoning depth and computational cost. Model merging has emerged as a promising, training-free technique to address this challenge by arithmetically combining the weights of a general-purpose model with a specialized reasoning model. While various merging techniques exist, their potential to create a spectrum of models with fine-grained control over reasoning abilities remains largely unexplored. This work presents a large-scale empirical study evaluating a range of model merging techniques across multiple reasoning benchmarks. We systematically vary merging strengths to construct accuracy-efficiency curves, providing the first comprehensive view of the tunable performance landscape. Our findings reveal that model merging offers an effective and controllable method for calibrating the trade-off between reasoning accuracy and token efficiency, even when parent models have highly divergent weight spaces. Crucially, we identify instances of Pareto Improvement, where a merged model achieves both higher accuracy and lower token consumption than one of its parents. Our study provides the first comprehensive analysis of this tunable space, offering practical guidelines for creating LLMs with specific reasoning profiles to meet diverse application demands.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
