通过思维链正确性感知蒸馏缓解问题与答案之间的虚假相关性
Mitigating Spurious Correlations Between Question and Answer via Chain-of-Thought Correctness Perception Distillation
AI总结：
针对SLM微调中CoT数据噪声导致问题与答案虚假相关的问题，提出CoPeD方法，通过正确性感知任务设置和加权损失提升推理质量，在IND和OOD数据集上验证有效。
摘要：
大型语言模型（LLMs）在推理任务上表现出色，但部署成本高昂。因此，小型语言模型（SLMs）在由LLMs生成的思维链（CoT）数据上进行微调，以复制LLMs的能力。然而，这些CoT数据可能包含噪声推理，这些推理要么无法证实答案，要么不提供支持答案预测的额外信息，这导致SLMs捕捉到问题与答案之间的虚假相关性，并损害推理质量。在这项工作中，我们提出了思维链正确性感知蒸馏（CoPeD），旨在从任务设置和数据利用的角度提高学生模型的推理质量。首先，我们引入了一种正确性感知的任务设置，鼓励学生模型基于正确的推理来预测答案，并在推理不正确时对其进行修正。这种设置提高了推理的忠实度，并允许模型从错误中学习。然后，我们提出了一种正确性感知加权损失，该损失根据推理和答案的组合损失动态调整每个训练实例的贡献。该策略鼓励模型更多地关注推理为正确答案提供更强支持的样本。实验表明，CoPeD在分布内（IND）和分布外（OOD）基准推理数据集上均有效。
英文摘要：
Large language models (LLMs) excel at reasoning tasks but are expensive to deploy. Thus small language models (SLMs) are fine-tuned on CoT data generated by LLMs to copy LLMs' abilities. However, these CoT data may include noisy rationales that either fail to substantiate the answers or contribute no additional information to support answer prediction, which leads SLMs to capture spurious correlations between questions and answers and compromise the quality of reasoning. In this work, we propose Chain-of-Thought Correctness Perception Distillation (CoPeD), which aims to improve the reasoning quality of the student model from the perspectives of task setting and data utilization. Firstly, we introduce a correctness-aware task setting that encourages the student model to predict answers based on correct rationales and revise them when they are incorrect. This setting improves the faithfulness of reasoning and allows the model to learn from its mistakes. Then, we propose a Correctness-Aware Weighted loss, which dynamically adjusts the contribution of each training instance based on the combined loss of the rationale and the answer. This strategy encourages the model to focus more on samples where the rationale offers stronger support for the correct answer. Experiments have shown that CoPeD is effective on both in-distribution (IND) and out-of-distribution (OOD) benchmark reasoning datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
