偏差增强一致性训练减少思维链中的有偏推理
Bias-Augmented Consistency Training Reduces Biased Reasoning in Chain-of-Thought
AI总结：
本文提出偏差增强一致性训练（BCT），通过无监督微调使模型在有偏与无偏提示间保持一致推理，在GPT-3.5-Turbo上将留出任务有偏推理率降低86%，并泛化至未知偏差。
摘要：
思维链提示（CoT）有潜力提升语言模型推理的可解释性。但CoT也可能系统性地歪曲影响模型行为的因素——例如，给出符合用户观点的合理化解释。我们首先创建了一个包含9种不同偏差的新数据集，这些偏差影响GPT-3.5-Turbo和Llama-8b模型。它们由伪少样本模式、事后合理化和谄媚设定组成。模型会转向偏差所暗示的答案，而在CoT中不提及偏差的影响。为缓解这一有偏推理问题，我们引入了偏差增强一致性训练（BCT），这是一种无监督微调方案，训练模型在有和没有偏差特征的提示之间给出一致的推理。我们构建了一套测试套件，在7个问答任务上测试9种形式的有偏推理，并发现将BCT应用于GPT-3.5-Turbo（使用一种偏差）可将留出任务上的有偏推理率降低86%。此外，该模型泛化到其他形式的偏差，将留出偏差上的有偏推理平均降低37%。由于BCT泛化到留出偏差且不需要金标签，该方法有望减少来自尚未知晓的偏差的有偏推理，以及在无法获得真实推理的任务中减少有偏推理。
英文摘要：
Chain-of-thought prompting (CoT) has the potential to improve the explainability of language model reasoning. But CoT can also systematically misrepresent the factors influencing models' behavior -- for example, rationalizing answers in line with a user's opinion. We first create a new dataset of 9 different biases that affect GPT-3.5-Turbo and Llama-8b models. These consist of spurious-few-shot patterns, post hoc rationalization, and sycophantic settings. Models switch to the answer implied by the bias, without mentioning the effect of the bias in the CoT. To mitigate this biased reasoning problem, we introduce bias-augmented consistency training (BCT), an unsupervised fine-tuning scheme that trains models to give consistent reasoning across prompts with and without biasing features. We construct a suite testing nine forms of biased reasoning on seven question-answering tasks, and find that applying BCT to GPT-3.5-Turbo with one bias reduces the rate of biased reasoning by 86\% on held-out tasks. Moreover, this model generalizes to other forms of bias, reducing biased reasoning on held-out biases by an average of 37\%. As BCT generalizes to held-out biases and does not require gold labels, this method may hold promise for reducing biased reasoning from as-of-yet unknown biases and on tasks where ground truth reasoning is unavailable.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
