ThinkSwitch:基于LoRA和权重插值的上下文蒸馏用于特定目的推理任务
ThinkSwitch: Context Distillation with LoRA and Weight Interpolation for Specific-Purpose Reasoning Tasks
AI总结:
提出ThinkSwitch方法,通过QLoRA蒸馏和球面权重插值协同训练指令模型和思考模型,在AIME 2026和PubMedQA上分别提升指令模型10/30→20/30和13/30→18/30,思考模型14/30→22/30和18/30→25/30,仅需15个训练提示和$2.86成本。
摘要:
大型语言模型通常通过在产生最终答案之前花费推理时间计算来改进困难任务。额外的计算可能有用,但也增加了延迟、令牌成本和部署复杂性。我们引入了 extbf{ThinkSwitch},一种低计算量的程序,用于协同训练配对的指令和思考检查点。从兼容的Qwen3-4B指令和思考模型开始,每次迭代要求思考检查点生成答案,移除推理轨迹,通过QLoRA将仅答案对蒸馏到指令检查点,并通过球面权重插值重建思考检查点。唯一的人工输入是任务提示;标签由模型自身生成。在30个问题的AIME 2026评估中,ThinkSwitch将指令检查点从10/30提升到20/30,思考检查点从14/30提升到22/30。在30个问题的PubMedQA子集上,它将指令检查点从13/30提升到18/30,思考检查点从18/30提升到25/30。完整实验每个领域使用15个训练提示,在单个云RTX 3070上花费2.86美元。结果规模较小,但表明有针对性的蒸馏循环可以将显式推理的部分好处转移到权重中,同时保留独立的思考模式。
英文摘要:
Large language models often improve on difficult tasks by spending inference-time compute on a reasoning trace before producing the final answer. That extra computation can be useful, but it also raises latency, token cost, and deployment complexity. We introduce \textbf{ThinkSwitch}, a low-compute procedure for co-training paired instruct and thinking checkpoints. Starting from compatible Qwen3-4B instruct and thinking models, each iteration asks the thinking checkpoint to generate answers, removes the reasoning trace, distills the answer-only pairs into the instruct checkpoint with QLoRA, and reconstructs a thinking checkpoint with spherical weight interpolation. The only human-supplied inputs are task prompts; the labels are generated by the model itself. On a 30-question AIME 2026 evaluation, ThinkSwitch improves the instruct checkpoint from 10/30 to 20/30 and the thinking checkpoint from 14/30 to 22/30. On a 30-question PubMedQA subset, it improves the instruct checkpoint from 13/30 to 18/30 and the thinking checkpoint from 18/30 to 25/30. The complete experiment uses 15 training prompts per domain and costs \$2.86 on a single cloud RTX 3070. The results are small-scale, but they indicate that targeted distillation loops can move part of the benefit of explicit reasoning into weights while preserving a separate thinking mode.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。