D-COT：为小语言模型高效推理的纪律性推理学习
D-COT: Disciplined Chain-of-Thought Learning for Efficient Reasoning in Small Language Models
AI总结：
D-CoT通过引入控制标签强制结构化推理，提升小语言模型在推理任务中的性能并减少计算成本。
摘要：
链式思维（CoT）从大语言模型（LLMs）的知识蒸馏常常导致小语言模型（SLMs）出现"过度思考"，从而引起性能下降和过多的token消耗。在本研究中，我们提出了纪律性链式思维（D-CoT），这是一种新的框架，通过控制标签--如<TEMP_LOW>用于事实核查和<TEMP_HIGH>用于多视角探索--作为训练期间的辅助支架，强制执行结构化的推理过程。通过优化CoT轨迹，D-CoT抑制了推理漂移，同时实现了token减少和性能提升。我们在Qwen3-8B上展示了我们的方法的有效性：仅使用5000个训练样本，D-CoT显著提升了GPQA-diamond上的准确率9.9%，并在MMLU-Pro（零样本）上提升了9.1%，同时大幅降低了计算成本。此外，我们确认模型内部化了这种纪律性思维结构，在推理过程中即使没有显式的控制标签也能保持高性能。
英文摘要：
Chain-of-Thought (CoT) distillation from Large Language Models (LLMs) often induces "overthinking" in Small Language Models (SLMs), leading to performance degradation and excessive token consumption. In this study, we propose Disciplined Chain-of-Thought (D-CoT), a novel framework that enforces a structured reasoning process using control tags -- such as <TEMP_LOW> for fact-checking and <TEMP_HIGH> for multi-perspective exploration -- as auxiliary scaffolding during training. By optimizing the CoT trajectory, D-CoT suppresses reasoning drift and simultaneously achieves token reduction and performance improvement. We demonstrate the efficacy of our approach on Qwen3-8B: with only 5,000 training samples, D-CoT significantly boosts accuracy on GPQA-diamond by 9.9% and MMLU-Pro (0-shot) by 9.1%, while drastically reducing computational costs. Furthermore, we confirm that the model internalizes this disciplined thought structure, maintaining high performance even without explicit control tags during inference.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
