防御性思维链：结构化推理激发大型语言模型对抗参考数据损坏的鲁棒性
Chain-of-Defensive-Thought: Structured Reasoning Elicits Robustness in Large Language Models against Reference Corruption
AI总结：
本研究提出防御性思维链方法，通过提供结构化防御推理示例，显著提升大型语言模型在非推理任务中对抗参考数据损坏的鲁棒性，有效缓解了提示注入攻击导致的准确率下降。
摘要：
思维链提示在促进大型语言模型的推理能力方面取得了巨大成功。在本研究中，我们探索如何利用这些增强的推理能力，来提升大型语言模型在非推理导向任务中的鲁棒性。特别是，我们展示了广泛的大型语言模型如何通过一种名为防御性思维链的简单方法，显著改善对抗参考数据损坏的鲁棒性，该方法仅提供少量包含结构化防御推理的示例作为演示。从经验上看，这些改进是惊人的，特别是考虑到该方法的简单性和适用性。例如，在Natural Questions任务中，当提供的10个参考中有1个被提示注入攻击损坏时，使用标准提示的GPT-4o准确率从60%降至低至3%。相比之下，使用防御性思维链提示的GPT-4o保持了50%的准确率。
英文摘要：
Chain-of-thought prompting has demonstrated great success in facilitating the reasoning abilities of large language models. In this work, we explore how these enhanced reasoning abilities can be exploited to improve the robustness of large language models in tasks that are not necessarily reasoning-focused. In particular, we show how a wide range of large language models exhibit significantly improved robustness against reference corruption using a simple method called chain-of-defensive-thought, where only a few exemplars with structured and defensive reasoning are provided as demonstrations. Empirically, the improvements can be astounding, especially given the simplicity and applicability of the method. For example, in the Natural Questions task, the accuracy of GPT-4o degrades from 60% to as low as 3% with standard prompting when 1 out of 10 references provided is corrupted with prompt injection attacks. In contrast, GPT-4o using chain-of-defensive-thought prompting maintains an accuracy of 50%.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
