Exploring the Universal Vulnerability of Prompt-based Learning Paradigm
Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京国家信息科学与技术研究中心);Huazhong University of Science and Technology(华中科技大学);Chongqing University(重庆大学)
Lei Xu, Yangyi Chen, Ganqu Cui, Hongcheng Gao, Zhiyuan Liu
Prompt-based learning paradigm bridges the gap between pre-training and fine-tuning, and works effectively under the few-shot setting. However, we find that this learning paradigm inherits the vulnerability from the pre-training stage, where model predictions can be misled by inserting certain triggers into the text. In this paper, we explore this universal vulnerability by either injecting backdoor triggers or searching for adversarial triggers on pre-trained language models using only plain text. In both scenarios, we demonstrate that our triggers can totally control or severely decrease the performance of prompt-based models fine-tuned on arbitrary downstream tasks, reflecting the universal vulnerability of the prompt-based learning paradigm. Further experiments show that adversarial triggers have good transferability among language models. We also find conventional fine-tuning models are not vulnerable to adversarial triggers constructed from pre-trained language models. We conclude by proposing a potential solution to mitigate our attack methods. Code and data are publicly available at https://github.com/leix28/prompt-universal-vulnerability
Vulnerability of Text-to-Image Models to Prompt Template Stealing: A Differential Evolution Approach
Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所);University of California at Merced(加州大学默塞德分校);University of Chinese Academy of Sciences(中国科学院大学);The University of Sydney(悉尼大学)
ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation
INSAIT, Sofia University "St. Kliment Ohridski"(INSAIT索菲亚大学"圣克莱门特·欧赫里迪斯基");Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, TU Darmstadt and National Research Center for Applied Cybersecurity ATHENE(无处不在知识处理实验室(UKP实验室)、计算机科学系、图腾达姆斯塔特大学和应用网络安全国家研究中心ATHENE);Arizona State University(亚利桑那州立大学)
Exploring the Vulnerability of the Content Moderation Guardrail in Large Language Models via Intent Manipulation
Boise State University(博伊州立大学);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Pittsburgh(匹兹堡大学);New York University(纽约大学);Florida International University(佛罗里达国际大学)
To Think or Not to Think: Exploring the Unthinking Vulnerability in Large Reasoning Models
The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Huawei International, Singapore(华为国际(新加坡));State University of New York at Buffalo(纽约州立大学布法罗分校)
MPrompt: Exploring Multi-level Prompt Tuning for Machine Reading Comprehension
Agency for Science, Technology and Research (A*STAR)(新加坡科技研究局);Osaka University(大阪大学);Meetyou AI Lab(美图AI实验室);University of Chinese Academy of Sciences(中国科学院大学);Xiamen Key Laboratory of Women’s Internet Health Management(厦门市女性互联网健康管理重点实验室)
Exploring prompts to elicit memorization in masked language model-based named entity recognition
University of Vienna(维也纳大学);Faculty of Computer Science, University of Vienna(维也纳大学计算机科学学院);UniVie Doctoral School Computer Science(维也纳大学计算机科学博士学院);Faculty of Philological and Cultural Studies, University of Vienna(维也纳大学语言与文化研究学院)