Dense retrieval systems have been widely used in various NLP applications. However, their vulnerabilities to potential attacks have been underexplored. This paper investigates a novel attack scenario where the attackers aim to mislead the retrieval system into retrieving the attacker-specified contents. Those contents, injected into the retrieval corpus by attackers, can include harmful text like hate speech or spam. Unlike prior methods that rely on model weights and generate conspicuous, unnatural outputs, we propose a covert backdoor attack triggered by grammar errors. Our approach ensures that the attacked models can function normally for standard queries while covertly triggering the retrieval of the attacker's contents in response to minor linguistic mistakes. Specifically, dense retrievers are trained with contrastive loss and hard negative sampling. Surprisingly, our findings demonstrate that contrastive loss is notably sensitive to grammatical errors, and hard negative sampling can exacerbate susceptibility to backdoor attacks. Our proposed method achieves a high attack success rate with a minimal corpus poisoning rate of only 0.048\%, while preserving normal retrieval performance. This indicates that the method has negligible impact on user experience for error-free queries. Furthermore, evaluations across three real-world defense strategies reveal that the malicious passages embedded within the corpus remain highly resistant to detection and filtering, underscoring the robustness and subtlety of the proposed attack \footnote{Codes of this work are available at https://github.com/ruyue0001/Backdoor_DPR.}.
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);Beihang University(北京航空航天大学);Huazhong University of Science and Technology(华中科技大学)
Merging Triggers, Breaking Backdoors: Defensive Poisoning for Instruction-Tuned Language Models
Graduate School of Artificial Intelligence, POSTECH, Republic of Korea(POSTECH人工智能研究生院,韩国);Department of Computer Science and Engineering, POSTECH, Republic of Korea(POSTECH计算机科学与工程系,韩国)
Triggerless Backdoor Attack for NLP Tasks with Clean Labels
Zhejiang University(浙江大学);Nanyang Technological University(南洋理工大学);Shanghai Institute for Advanced Study of Zhejiang University(浙江大学上海高等研究院);Shanghai AI Laboratory(上海人工智能实验室);Chongqing University(重庆大学);Peng Cheng Laboratory(鹏城实验室);Peking University(北京大学)