思维链依然思考得快：APriCoT 帮助实现慢思考
Chain of Thought Still Thinks Fast: APriCoT Helps with Thinking Slow
AI总结：
针对语言模型在MMLU任务中因吸收训练数据偏差而影响答案选择的问题，提出了反事实提示方法APriCoT，有效降低了基础概率影响并提升了整体准确率，实现了更稳健公平的慢思考。
摘要：
语言模型会从训练数据中吸收偏差，导致其预测由统计规律而非语义相关性驱动。我们调查了这些偏差对 Massive Multi-Task Language Understanding (MMLU) 任务中答案选择偏好的影响。研究结果表明，这些偏差能够预测模型偏好，并反映了人类的应试策略，即使在使用思维链 (CoT) 推理时也是如此。为了解决这一问题，我们引入了带有不可知启动 CoT 的反事实提示方法 (APriCoT)。我们证明，仅使用带有 CoT 的反事实提示不足以缓解偏差，而 APriCoT 能有效降低基础概率的影响，同时提高整体准确率。我们的结果表明，缓解偏差需要一个慢思考过程，仅靠 CoT 可能无法提供这一点，因为在某些提示方法下它往往会强化快思考模型偏差。APriCoT 是朝着开发能够进行慢思考的更稳健、更公平的语言模型迈出的一步。
英文摘要：
Language models are known to absorb biases from their training data, leading to predictions driven by statistical regularities rather than semantic relevance. We investigate the impact of these biases on answer choice preferences in the Massive Multi-Task Language Understanding (MMLU) task. Our findings show that these biases are predictive of model preference and mirror human test-taking strategies even when chain of thought (CoT) reasoning is used. To address this issue, we introduce Counterfactual Prompting with Agnostically Primed CoT (APriCoT). We demonstrate that while Counterfactual Prompting with CoT alone is insufficient to mitigate bias, APriCoT effectively reduces the influence of base-rate probabilities while improving overall accuracy. Our results suggest that mitigating bias requires a slow thinking process which CoT alone may not provide as it tends to reinforce fast thinking model bias under some prompting methodologies. APriCoT is a step toward developing more robust and fair language models that can think slow.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
