Large language models (LLMs) have shown state-of-the-art results in translating natural language questions into SQL queries (Text-to-SQL), a long-standing challenge within the database community. However, security concerns remain largely unexplored, particularly the threat of backdoor attacks, which can introduce malicious behaviors into models through fine-tuning with poisoned datasets. In this work, we systematically investigate the vulnerabilities of LLM-based Text-to-SQL models and present ToxicSQL, a novel backdoor attack framework. Our approach leverages stealthy {semantic and character-level triggers} to make backdoors difficult to detect and remove, ensuring that malicious behaviors remain covert while maintaining high model accuracy on benign inputs. Furthermore, we propose leveraging SQL injection payloads as backdoor targets, enabling the generation of malicious yet executable SQL queries, which pose severe security and privacy risks in language model-based SQL development. We demonstrate that injecting only 0.44% of poisoned data can result in an attack success rate of 79.41%, posing a significant risk to database security. Additionally, we propose detection and mitigation strategies to enhance model reliability. Our findings highlight the urgent need for security-aware Text-to-SQL development, emphasizing the importance of robust defenses against backdoor threats.
Injection, Attack and Erasure: Revocable Backdoor Attacks via Machine Unlearning
Engineering Research Center of Transportation Information and Safety (ERCTIS), MoE of China, School of Computer Science and Artificial Intelligence, Wuhan University of Technology(交通运输信息与安全工程研究中心(ERCTIS)、中国教育部、武汉理工大学计算机科学与人工智能学院)
When Prompts Become Payloads: A Framework for Mitigating SQL Injection Attacks in Large Language Model-Driven Applications
Hasso Plattner Institute, University of Potsdam, Germany(波茨坦大学霍斯曼-普拉特研究所, 德国);Chemnitz University of Technology, Chemnitz, Germany(Chemnitz技术大学, Chemnitz, 德国)
TPSQLi: Test Prioritization for SQL Injection Vulnerability Detection in Web Applications
National Taiwan University(台湾大学);CyberLink Corporation(网路家庭资讯事业股份有限公司);National Yang Ming Chiao Tung University(国立阳明交通大学);National Dong Hwa University(国立东华大学)
Flareon: Stealthy any2any Backdoor Injection via Poisoned Augmentation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences(中国科学院深圳先进技术研究院);University of Chinese Academy of Sciences(中国科学院大学);University of Cambridge(剑桥大学);University of Macau(澳门大学)
DBIA: Data-free Backdoor Injection Attack against Transformer Networks
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);University of Chinese Academy of Sciences(中国科学院大学);Boston University(波士顿大学)
College of Control Science and Engineering, China University of Petroleum (East China)(控制科学与工程学院,中国石油大学(华东));College of Computer Science and Electronic Engineering, Hunan University(计算机科学与电子工程学院,湖南大学)
STAIN-FL: Stealthy Targeted Attack Injection with Contextual Triggers in Federated Learning
Singapore Institute of Technology (SIT)(新加坡理工学院(SIT));SingHealth Duke-NUS AI in Medicine Institute(新加坡保健服务集团杜克-新加坡国立大学医学院AI研究所);SingHealth AI Office(新加坡保健服务集团AI办公室)
BadDLM: Backdooring Diffusion Language Models with Diverse Targets
National University of Singapore(新加坡国立大学);Peking University(北京大学);Shanghai Jiao Tong University(上海交通大学);Jiutian Research(中移九天);Universal Database(通用数据库);Tsinghua University(清华大学)
Ouroboros: Self-Referential Backdoor Attacks on Speech Enhancement via Clean Audio Triggers
Faculty of Electrical Engineering and Computer Science, Ningbo University(宁波大学电气工程与计算机科学学院);College of Artificial Intelligence, Ningbo University of Finance and Economics(宁波财经学院人工智能学院)
Towards Backdoor Stealthiness in Model Parameter Space
Radboud University(拉德博德大学);Delft University of Technology(代尔夫特理工大学);Radboud University, Nijmegen, The Netherlands(拉德博德大学);University of Zagreb Faculty of Electrical Engineering and Computing(Zagreb 工程与计算学院)
State Key Laboratory of Integrated Services Networks (ISN)(综合业务网理论及关键技术国家重点实验室);Shaanxi Key Laboratory of Network and System Security, Xidian University(西安电子科技大学陕西省网络与系统安全重点实验室);School of Data Science and Engineering, East China Normal University(华东师范大学数据科学与工程学院);Ant Group(蚂蚁集团);College of Computer and Data Science, Fuzhou University(福州大学计算机与数据科学学院)
ModSec-AdvLearn: Countering Adversarial SQL Injections with Robust Machine Learning
Dept. of Electrical and Electronic Engineering, University of Cagliari(电子工程系,卡利亚里大学);Department of Computer, Control and Management Engineering, Sapienza University(计算机、控制与管理工程系,萨皮恩扎大学);Dept. of Digital Security, EURECOM(数字安全系,EURECOM);Department of Informatics, Bioengineering, Robotics and Systems Engineering (DIBRIS), University of Genova(信息学、生物工程、机器人学和系统工程系(DIBRIS),热那亚大学);Prima Assicurazioni(普里玛保险);Endor Labs(Endor实验室);Pluribus One