haystack中的触发器：提取和重建LLM后门触发器
The Trigger in the Haystack: Extracting and Reconstructing LLM Backdoor Triggers
AI总结：
本文提出了一种无需先验知识的LLM后门触发器扫描方法，通过记忆提取和输出分布分析来检测和恢复后门触发器。
摘要：
检测模型是否被污染是人工智能安全领域长期存在的问题。在本工作中，我们提出了一种实用的扫描器，用于识别因果语言模型中的睡眠代理式后门。我们的方法基于两个关键发现：首先，睡眠代理倾向于记忆污染数据，这使得可以利用记忆提取技术泄露后门示例。其次，被污染的LLM在输入存在后门触发器时，其输出分布和注意力头会表现出独特的模式。基于这些观察，我们开发了一种可扩展的后门扫描方法，该方法不假定对触发器或目标行为的任何先验知识，仅需推理操作即可实现。我们的扫描器能够自然地融入更广泛的防御策略，且不改变模型性能。我们证明了我们的方法能够在多种后门场景和广泛模型和微调方法中恢复有效的触发器。
英文摘要：
Detecting whether a model has been poisoned is a longstanding problem in AI security. In this work, we present a practical scanner for identifying sleeper agent-style backdoors in causal language models. Our approach relies on two key findings: first, sleeper agents tend to memorize poisoning data, making it possible to leak backdoor examples using memory extraction techniques. Second, poisoned LLMs exhibit distinctive patterns in their output distributions and attention heads when backdoor triggers are present in the input. Guided by these observations, we develop a scalable backdoor scanning methodology that assumes no prior knowledge of the trigger or target behavior and requires only inference operations. Our scanner integrates naturally into broader defensive strategies and does not alter model performance. We show that our method recovers working triggers across multiple backdoor scenarios and a broad range of models and fine-tuning methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
