对思维链推理的先发答案“攻击”
Preemptive Answer "Attacks" on Chain-of-Thought Reasoning
AI总结：
本文提出“先发答案”场景，揭示LLM在推理前获得答案会显著削弱其CoT推理能力，并提出两项缓解措施以增强鲁棒性。
摘要：
大型语言模型（LLMs）与思维链（CoT）提示结合时展现出令人印象深刻的推理能力。然而，该方法的鲁棒性值得进一步研究。本文引入了一种称为先发答案的新场景，即LLM在进行推理之前就获得了一个答案。这种情况可能无意中发生，或由恶意用户通过提示注入攻击引发。实验表明，先发答案在各种CoT方法和广泛的数据集上显著损害了模型的推理能力。为了增强推理的鲁棒性，我们提出了两项在一定程度上缓解此问题的措施。
英文摘要：
Large language models (LLMs) showcase impressive reasoning capabilities when coupled with Chain-of-Thought (CoT) prompting. However, the robustness of this approach warrants further investigation. In this paper, we introduce a novel scenario termed preemptive answers, where the LLM obtains an answer before engaging in reasoning. This situation can arise inadvertently or induced by malicious users by prompt injection attacks. Experiments reveal that preemptive answers significantly impair the model's reasoning capability across various CoT methods and a broad spectrum of datasets. To bolster the robustness of reasoning, we propose two measures aimed at mitigating this issue to some extent.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
