评估与保障基于检索的上下文学习的对抗鲁棒性
Evaluating and Safeguarding the Adversarial Robustness of Retrieval-Based In-Context Learning
AI总结：
该研究针对检索增强型上下文学习的对抗鲁棒性展开评估，发现其对测试样本攻击鲁棒性提升但易受示范攻击影响，并提出免训练防御方法DARD，可将攻击成功率较基线降低15%。
摘要：
随着LLaMA、OpenAI GPT-3等大语言模型的出现，上下文学习（In-Context Learning, ICL）因其有效性与高效性受到广泛关注。但ICL对提示中示范样本的选择、排序以及用于编码示范的表述器（verbaliser）高度敏感。检索增强型ICL方法试图通过利用检索器提取语义相关的示例作为示范来解决这一问题。尽管该方法能带来更准确的结果，但其针对各类对抗攻击的鲁棒性仍未得到充分研究，这些攻击类型包括测试样本扰动、示范样本扰动以及检索数据扰动。\n我们的研究表明，检索增强模型能够提升针对测试样本攻击的鲁棒性，相比 vanilla ICL（原生上下文学习），攻击成功率（Attack Success Rate, ASR）降低4.87%；但这类模型对示范样本存在过度信任，导致针对示范样本攻击的ASR上升2%。对抗训练有助于提升ICL方法对对抗攻击的鲁棒性，但在大语言模型场景下，这类训练方案的成本过高。作为替代方案，我们提出了一种有效的免训练对抗防御方法DARD，该方法通过受攻击样本丰富示例池。实验证明，DARD能同时提升性能与鲁棒性，相比基线方法ASR降低15%。代码与数据已公开发布以推动后续研究：https://github.com/simonucl/adv-retreival-icl
英文摘要：
With the emergence of large language models, such as LLaMA and OpenAI GPT-3, In-Context Learning (ICL) gained significant attention due to its effectiveness and efficiency. However, ICL is very sensitive to the choice, order, and verbaliser used to encode the demonstrations in the prompt. Retrieval-Augmented ICL methods try to address this problem by leveraging retrievers to extract semantically related examples as demonstrations. While this approach yields more accurate results, its robustness against various types of adversarial attacks, including perturbations on test samples, demonstrations, and retrieved data, remains under-explored. Our study reveals that retrieval-augmented models can enhance robustness against test sample attacks, outperforming vanilla ICL with a 4.87% reduction in Attack Success Rate (ASR); however, they exhibit overconfidence in the demonstrations, leading to a 2% increase in ASR for demonstration attacks. Adversarial training can help improve the robustness of ICL methods to adversarial attacks; however, such a training scheme can be too costly in the context of LLMs. As an alternative, we introduce an effective training-free adversarial defence method, DARD, which enriches the example pool with those attacked samples. We show that DARD yields improvements in performance and robustness, achieving a 15% reduction in ASR over the baselines. Code and data are released to encourage further research: https://github.com/simonucl/adv-retreival-icl
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
