Large Language Models (LLMs), especially those accessed via APIs, have demonstrated impressive capabilities across various domains. However, users without technical expertise often turn to (untrustworthy) third-party services, such as prompt engineering, to enhance their LLM experience, creating vulnerabilities to adversarial threats like backdoor attacks. Backdoor-compromised LLMs generate malicious outputs to users when inputs contain specific "triggers" set by attackers. Traditional defense strategies, originally designed for small-scale models, are impractical for API-accessible LLMs due to limited model access, high computational costs, and data requirements. To address these limitations, we propose Chain-of-Scrutiny (CoS) which leverages LLMs' unique reasoning abilities to mitigate backdoor attacks. It guides the LLM to generate reasoning steps for a given input and scrutinizes for consistency with the final output -- any inconsistencies indicating a potential attack. It is well-suited for the popular API-only LLM deployments, enabling detection at minimal cost and with little data. User-friendly and driven by natural language, it allows non-experts to perform the defense independently while maintaining transparency. We validate the effectiveness of CoS through extensive experiments on various tasks and LLMs, with results showing greater benefits for more powerful LLMs.
Proactive Adversarial Defense: Harnessing Prompt Tuning in Vision-Language Models to Detect Unseen Backdoored Images
University of West Florida(西佛罗里达大学);Florida Institute For Human and Machine Cognition (IHMC)(佛罗里达人机认知研究所);Center for Cybersecurity, University of West Florida(西佛罗里达大学网络安全中心)
Backdoor Samples Detection Based on Perturbation Discrepancy Consistency in Pre-trained Language Models
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University(航天信息安全部门、教育部、网络安全科学与工程学院、武汉大学)
Mark Your LLM: Detecting the Misuse of Open-Source Large Language Models via Watermarking
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));Tsinghua University(清华大学);The Hong Kong University of Science and Technology(香港科技大学)
DeBackdoor: A Deductive Framework for Detecting Backdoor Attacks on Deep Models with Limited Data
Qatar Computing Research Institute, Hamad Bin Khalifa University(卡塔尔计算研究所,哈马德·本·哈利法大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学)
FreeEagle: Detecting Complex Neural Trojans in Data-Free Cases
Zhejiang University(浙江大学);Pennsylvania State University(宾夕法尼亚州立大学);Chinese Aeronautical Establishment(中国航空研究院);National University of Defense Technology(国防科技大学)
Detecting Backdoors During the Inference Stage Based on Corruption Robustness Consistency
Huazhong University of Science and Technology(华中科技大学);National Engineering Research Center for Big Data Technology and System(大数据技术与系统国家工程研究中心);Wuhan University(武汉大学)
Isolate Trigger: Detecting and Eliminating Adaptive Backdoor Attacks
Beijing University of Posts and Telecommunications(北京邮电大学);China Mobile Research Institute(中国移动研究院);University of Technology Sydney(悉尼科技大学);Nanjing University of Science and Technology(南京理工大学)
DualSentinel: A Lightweight Framework for Detecting Targeted Attacks in Black-box LLM via Dual Entropy Lull Pattern
The Hong Kong University of Science and Technology(香港科学与技术大学);The State Key Laboratory of Blockchain and Data Security(区块链与数据安全国家重点实验室);School of Cyber Science and Technology(网络安全科学与技术学院)
DMGNN: Detecting and Mitigating Backdoor Attacks in Graph Neural Networks
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Yangzhou University(扬州大学);University of Southern Queensland(南昆士兰大学);Data61, CSIRO(联邦科学与工业研究组织Data61);University of Technology Sydney(悉尼科技大学)
Detecting Hidden Behaviors in LLMs via Activation-matched Finetuning
Bonn-Aachen International Center for Information Technology, University of Bonn(波恩大学波恩-亚琛国际信息技术中心);Lamarr Institute for Machine Learning and Artificial Intelligence(拉马尔机器学习与人工智能研究院)
Model X-ray:Detecting Backdoored Models via Decision Boundary
University of Science and Technology of China(中国科学技术大学);CFAR(计算与算法研究中心);IHPC(高性能计算研究院);A*STAR(新加坡科技研究局);National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学)
本文提出 Model X-ray,一种仅利用干净输入硬标签构建二维决策边界来检测后门模型并识别目标攻击标签的轻量级方法,无需攻击信息或模型参数先验,实验证明其在不同攻击、数据集和架构上均有效且高效。
TED-LaST: Towards Robust Backdoor Defense Against Adaptive Attacks
Deakin University(德肯大学);Beijing Normal-Hong Kong Baptist University(北京师范大学-香港 Baptist大学);University of New South Wales(新南威尔士大学);Griffith University(格里菲斯大学)
Shortcuts Everywhere and Nowhere: Exploring Multi-Trigger Backdoor Attacks
Shanghai Key Lab of Intell. Info. Processing, School of CS, Fudan University(上海智能信息处理关键实验室,复旦大学计算机学院);Bosch Research Asia Pacific & Bosch Center for Artificial Intelligence (BCAI)(博世亚太研究部及博世人工智能中心);School of Computing and Information Systems, the University of Melbourne(墨尔本大学计算机与信息学院);School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算机与信息学院)