相关性错觉：任意内容注入攻击欺骗检索器、重排序器和LLM判断者
Illusions of Relevance: Arbitrary Content Injection Attacks Deceive Retrievers, Rerankers, and LLM Judges
AI总结：
本文揭示了任意内容注入攻击可欺骗检索器、重排序器和LLM判断者，指出模型在安全性和鲁棒性上的弱点，并呼吁进一步研究。
摘要：
本文考虑了一种黑盒威胁模型，其中攻击者试图在搜索中传播任意不相关的内容。我们展示了检索器、重排序器和LLM相关性判断者都高度易受攻击，这些攻击能够使任意内容进入搜索结果的顶部并被赋予完美的相关性评分。我们研究了攻击者如何通过内容注入实现这一点，即在相关段落中注入任意句子，或在任意段落中注入查询术语。我们的研究分析了模型类别和大小、相关和不相关内容的平衡、注入位置、注入内容的毒性及严重程度以及LLM生成内容的作用等因素如何影响攻击成功率，得出新颖、令人担忧且往往反直觉的结果。我们的结果揭示了嵌入模型、基于LLM的评分模型和生成式LLM的弱点，引发了对语言模型通用鲁棒性、安全性和可信度的担忧，无论模型类型或其应用角色如何。我们还强调了对这些攻击进行稳健防御的挑战。分类器和更仔细提示的LLM判断者往往无法识别含有内容注入的段落，尤其是在考虑多样化的文本主题和风格时。我们的发现突显了对任意内容注入攻击进行进一步研究的必要性。我们释放了我们的代码供进一步研究。
英文摘要：
This work considers a black-box threat model in which adversaries attempt to propagate arbitrary non-relevant content in search. We show that retrievers, rerankers, and LLM relevance judges are all highly vulnerable to attacks that enable arbitrary content to be promoted to the top of search results and to be assigned perfect relevance scores. We investigate how attackers may achieve this via content injection, injecting arbitrary sentences into relevant passages or query terms into arbitrary passages. Our study analyzes how factors such as model class and size, the balance between relevant and non-relevant content, injection location, toxicity and severity of injected content, and the role of LLM-generated content influence attack success, yielding novel, concerning, and often counterintuitive results. Our results reveal a weakness in embedding models, LLM-based scoring models, and generative LLMs, raising concerns about the general robustness, safety, and trustworthiness of language models regardless of the type of model or the role in which they are employed. We also emphasize the challenges of robust defenses against these attacks. Classifiers and more carefully prompted LLM judges often fail to recognize passages with content injection, especially when considering diverse text topics and styles. Our findings highlight the need for further research into arbitrary content injection attacks. We release our code for further study.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
