arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.13532cs.CL
收藏

通过公开且非故意触发的后门攻击密集检索系统

Backdoor Attacks on Dense Retrieval via Public and Unintentional Triggers

Nanyang Technological University, Singapore(南洋理工大学);Zhejiang University(浙江大学);The Chinese University of Hong Kong(香港中文大学)

Quanyu Long, Yue Deng, LeiLei Gan, Wenya Wang, Sinno Jialin Pan

AI总结:

针对密集检索系统的脆弱性,本文提出一种由语法错误触发的隐蔽后门攻击,利用对比损失对语法错误的敏感性和难负样本采样的易感性,以极低语料投毒率实现高攻击成功率,且对正常查询无影响,恶意内容难被防御策略检测。

摘要:

密集检索系统已广泛应用于各种自然语言处理(NLP)应用中。然而,其对潜在攻击的脆弱性尚未得到充分研究。本文研究了一种新型攻击场景:攻击者旨在误导检索系统检索其指定的内容。这些由攻击者注入检索语料库的内容可能包含仇恨言论或垃圾邮件等有害文本。与依赖模型权重并生成显眼、不自然输出的现有方法不同,我们提出一种由语法错误触发的隐蔽后门攻击。该方法确保受攻击模型在标准查询下正常工作,同时在面对微小语言错误时隐蔽触发对攻击者内容的检索。具体而言,密集检索模型通过对比损失和难负样本采样训练。令人惊讶的是,我们发现对比损失对语法错误异常敏感,而难负样本采样会加剧对后门攻击的易感性。我们提出的方法在仅0.048%的语料投毒率下实现了高攻击成功率,同时保留正常检索性能,这表明该方法对无错误查询的用户体验影响可忽略不计。此外,在三种真实世界防御策略下的评估显示,嵌入语料库中的恶意段落仍极难被检测和过滤,凸显了所提攻击的鲁棒性和隐蔽性。

英文摘要:

Dense retrieval systems have been widely used in various NLP applications. However, their vulnerabilities to potential attacks have been underexplored. This paper investigates a novel attack scenario where the attackers aim to mislead the retrieval system into retrieving the attacker-specified contents. Those contents, injected into the retrieval corpus by attackers, can include harmful text like hate speech or spam. Unlike prior methods that rely on model weights and generate conspicuous, unnatural outputs, we propose a covert backdoor attack triggered by grammar errors. Our approach ensures that the attacked models can function normally for standard queries while covertly triggering the retrieval of the attacker's contents in response to minor linguistic mistakes. Specifically, dense retrievers are trained with contrastive loss and hard negative sampling. Surprisingly, our findings demonstrate that contrastive loss is notably sensitive to grammatical errors, and hard negative sampling can exacerbate susceptibility to backdoor attacks. Our proposed method achieves a high attack success rate with a minimal corpus poisoning rate of only 0.048\%, while preserving normal retrieval performance. This indicates that the method has negligible impact on user experience for error-free queries. Furthermore, evaluations across three real-world defense strategies reveal that the malicious passages embedded within the corpus remain highly resistant to detection and filtering, underscoring the robustness and subtlety of the proposed attack \footnote{Codes of this work are available at https://github.com/ruyue0001/Backdoor_DPR.}.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑