Large Language Models(LLMs) have been successful in numerous fields. Alignment has usually been applied to prevent them from harmful purposes. However, aligned LLMs remain vulnerable to jailbreak attacks that deliberately mislead them into producing harmful outputs. Existing jailbreaks are either black-box, using carefully crafted, unstealthy prompts, or white-box, requiring resource-intensive computation. In light of these challenges, we introduce ShallowJail, a novel attack that exploits shallow alignment in LLMs. ShallowJail can misguide LLMs' responses by manipulating the initial tokens during inference. Through extensive experiments, we demonstrate the effectiveness of ShallowJail, which substantially degrades the safety of state-of-the-art LLM responses. Our code is available at https://github.com/liuup/ShallowJail.
Graph-Regularized Sparse Autoencoders for LLM Safety Steering
ELLIS Institute Tübingen(图宾根ELLIS研究所);Max Planck Institute for Intelligent Systems(智能系统马克斯·普朗克研究所);Intesa Sanpaolo(Intesa Sanpaolo公司);University of Southern California(南加州大学)
Jailbreaking the Matrix: Nullspace Steering for Controlled Model Subversion
Department of Computer & Information Science & Engineering, University of Florida(佛罗里达大学计算机与信息科学与工程系);School of Computer Science, University of Oklahoma(俄克拉荷马大学计算机科学学院);Computer Science Laboratory, SRI International(SRI国际计算机科学实验室)
Bleeding Pathways: Vanishing Discriminability in LLM Hidden States Fuels Jailbreak Attacks
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院);Ant Group(蚂蚁集团)
SafeSteer: Adaptive Subspace Steering for Efficient Jailbreak Defense in Vision-Language Models
Nanjing University of Science and Technology(南京理工大学);School of Cyber Science and Engineering, Nanjing University of Science and Technology(南京理工大学网络空间安全学院);Nanyang Technological University(南洋理工大学);National University of Singapore(新加坡国立大学)
Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation
University of Technology Sydney, Sydney, Australia(技术悉尼大学);School of Cyber Science and Engineering, Southeast University, Nanjing, China(东南大学计算机科学与工程学院);Xi'an Jiaotong University, Xi'an, China(西安交通大学)
Renmin University of China(中国人民大学);Xi’an Jiaotong University(西安交通大学);University of Science and Technology of China(中国科学技术大学);Wuhan University(武汉大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室)
Agent MechSuits: Mechanistic Subspace Safety Steering for Multi-Turn CLI Agents
University of Georgia(佐治亚大学);University of South Florida(南佛罗里达大学);Rutgers University(罗格斯大学);University of Southern California(南加州大学);Johns Hopkins University(约翰霍普金斯大学)
Activation-Guided Local Editing for Jailbreaking Attacks
Beihang University(北航);Hangzhou Innovation Institute(杭州创新研究院);HKUST(香港科技大学);South China University of Technology(华南理工大学);Nanyang Technological University(南洋理工大学);Kunming University of Science and Technology(昆明理工大学)
MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs
School of Computer Science and Technology, East China Normal University(东华大学计算机科学与技术学院);Shanghai AI Lab, Shanghai Innovation Institute(上海人工智能实验室,上海创新研究院)
Probing the Robustness of Large Language Models Safety to Latent Perturbations
Tsinghua Shenzhen International Graduate School, Tsinghua University(清华大学深圳国际 Graduate School, 清华大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Fudan University(复旦大学);The University of Hong Kong(香港大学)
Compiling Activation Steering into Weights via Null-Space Constraints for Stealthy Backdoors
Zhejiang University(浙江大学);Palo Alto Networks(帕洛阿尔托网络);Ningbo Global Innovation Center, Zhejiang University(宁波全球创新中心,浙江大学);OPPO Research Institute(OPPO研究院);Qilu University of Technology(齐鲁大学)
Internal Activation Revision: Safeguarding Vision Language Models Without Parameter Update
Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);Fraunhofer FOKUS(弗劳恩霍夫协会FOKUS研究所);The University of Tokyo(东京大学);University of Alberta(阿尔伯塔大学)
Representation Bending for Large Language Model Safety
Seoul National University(首尔国立大学);Yonsei University(延世大学);AIM Intelligence(AIM智能);University of Michigan(密歇根大学);Stanford University(斯坦福大学);Amazon AWS(亚马逊AWS)