增强小模型思维链推理的泛化能力
Enhancing Generalization in Chain of Thought Reasoning for Smaller Models
AI总结：
针对小语言模型CoT推理泛化能力弱的问题，提出PRADA提示辅助域对抗微调框架，通过恢复域不变特征、增强提示域适应性提升泛化性，性能优于现有最优方法且可解释性更强。
摘要：
小语言模型中的思维链（Chain-of-Thought, CoT）推理是一项具有挑战性的自然语言处理问题，但在许多实际应用中需求极高。现有的CoT知识蒸馏方法往往导致小大语言模型（LLM）过度保守地记忆，泛化置信度较低。由于完全保留教师模型的CoT能力是不可能的，我们假设对抗性CoT微调对于开发具备鲁棒CoT泛化能力的小LLM至关重要。为此，我们提出了\textit{PRompt-Assisted Domain-Adversarial fine-tuning}（PRADA，提示辅助域对抗微调），这是一个整合多类CoT域的原则性微调框架。具体而言，PRADA在小LLM的CoT优化上实现了两项突破：（1）通过域对抗微调，恢复蒸馏过程中通常会丢失的域不变特征洞察；（2）采用域对抗方法增强CoT提示工程的域适应性。我们从理论上证明了该方法的有效性，并通过实验表明，它在广泛的任务中显著优于现有最优方法。此外，实验结果显示，使用PRADA的小LLM与域知识高度契合，从而提升了方法的可解释性。
英文摘要：
Chain-of-Thought (CoT) reasoning in smaller language models is a challenging natural language process problem yet highly desirable in many real-life applications. Existing CoT knowledge distillation methods often suffer from overly conservative memorization in smaller LLMs, leading to low generalization confidence. As fully preserving the CoT ability of teacher model is impossible, we hypothesize that adversarial CoT fine-tuning is crucial for developing smaller LLM with robust CoT generalization. To this end, we propose \textit{PRompt-Assisted Domain-Adversarial fine-tuning} (PRADA), a principled fine-tuning framework that integrates diverse CoT domains. Specifically, PRADA pioneers two CoT improvements in smaller LLM: (1) Recovering the domain-invariant feature insight which typically lost during distillation with domain adversarial fine-tuning; (2) Enhancing the domain adaptability of CoT prompt engineering by employing domain-adversarial approaches. We theoretically demonstrate the effectiveness of our approach and empirically show that it significantly outperforms the state of the arts in a wide range of tasks. Moreover, our empirical findings reveal that the smaller LLM, when leveraging PRADA, aligns closely with domain knowledge, thereby improving the explainability of our approach.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
