Cordyceps: 通过数据投毒对LLM的隐蔽控制攻击
Cordyceps: Covert Control Attacks on LLMs via Data Poisoning
AI总结：
提出一种数据投毒方法，通过语义关联教LLM隐藏任意恶意指令，实现隐蔽控制攻击，绕过多种防御。
摘要：
大型语言模型（LLM）通常在没有经过精心筛选的文本数据集上进行微调，而对手可以对这些数据集进行投毒。现有的投毒攻击主要依赖于固定的触发短语，而异常检测、干净数据正则化或在线监控等防御措施可以中和这些触发短语。在本文中，我们提出了一种数据投毒方法，通过共享知识（如事实或概念）与攻击者选择的短语之间的语义关联，可靠且隐蔽地教LLM一种信息隐藏方案。诱导的隐藏方案可以编码和解码任意恶意指令，从而揭示了一种新的、微妙的投毒诱导漏洞：隐蔽控制攻击。我们精确描述了隐蔽控制攻击，并在5个LLM、3个后门防御和4个提示注入防御上进行了评估。在少量投毒样本的情况下，隐蔽控制攻击在平均攻击成功率上比基于启发式的提示注入攻击高出约40%（相对于干净微调模型）。它们还绕过了基于检测和微调的防御，在后门防御后保持高达93%的攻击成功率，在提示注入防御后保持高达98%的攻击成功率。
英文摘要：
Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison. Existing poisoning attacks primarily rely on fixed trigger phrases that defenses such as outlier detection, clean-data regularization, or online monitoring can neutralize. In this paper, we propose a data poisoning method that teaches an LLM an information hiding scheme reliably and stealthily through semantic associations between shared knowledge such as facts or concepts and attacker-chosen phrases. The induced hiding scheme can encode and decode arbitrary malicious instructions, thus revealing a new and subtle poisoning-induced vulnerability: covert control attacks. We precisely characterize covert control attacks and evaluate them across $5$ LLMs, $3$ backdoor defenses, and $4$ prompt injection defenses. With a small poisoned fraction, covert control attacks outperform heuristic-based prompt injection attacks in average attack success rate by about $40\%$ relative to clean fine-tuned models. They also circumvent defenses based on detection and fine-tuning, maintaining up to $93\%$ attack success rate after backdoor defenses and up to $98\%$ after prompt injection defenses.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
