Evaluating and mitigating a generative system's susceptibility to jailbreak attacks is critical to its safe deployment. Given the number of deployable systems, full per-configuration evaluation and optimization is impractical. In this paper, we formalize the behavioral geometry of a population of models that, by leveraging previously evaluated and defended models, supports both efficient susceptibility prediction and effective defense transfer across a population. We apply the framework to 82 models spanning 24 providers and to 100 system configurations of a single base model. Simple methods that use the behavioral geometry reach an AUPRC of $0.94$ for susceptibility detection with $\approx98\%$ fewer probes relative to a full evaluation. Using the behavioral geometry to select which model to transfer an optimized defense from outperforms same-provider assignment ($+4.1\%$, $p = 0.004$) at no additional probe cost, with a set of three models sufficient to cover the population. Results are robust to hyperparameter selection and judge.
From Hallucinations to Jailbreaks: Rethinking the Vulnerability of Large Foundation Models
School of Information Sciences University of Illinois at Urbana-Champaign(信息科学学院伊利诺伊大学厄巴纳-香槟分校);Computer Science and Engineering HKUST(计算机科学与工程香港科技大学);Computer Science Department University of California, Los Angeles(计算机科学系加州大学洛杉矶分校);Research Scientist, Intel Labs(英特尔实验室研究员);School of Information Sciences University of Illinois Urbana-Champaign(信息科学学院伊利诺伊大学厄巴纳-香槟分校)
JALMBench: Benchmarking Jailbreak Vulnerabilities in Audio Language Models
The Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州));State Key Laboratory of Internet Architecture, Tsinghua University(清华大学互联网架构国家重点实验室);University of North Texas(北卡罗来纳州立大学);University of Science and Technology of China(中国科学技术大学);Fujian Normal University(福建师范大学);Nanjing University of Aeronautics and Astronautics(南京航空航天大学)
Jailbreaking and Mitigation of Vulnerabilities in Large Language Models
Hunan University Changsha, PRC;Georgia Institute of Technology Atlanta, USA;Kyoto University Kyoto, Japan;Purdue University West Lafayette, USA;National Taiwan Normal University Taipei, ROC;University of Liverpool Suzhou, PRC;Hong Kong University of Science;University of Hawaii Honolulu, USA;The University of Texas at Dallas Dallas, USA;University of Wisconsin-Madison Madison, USA;Emory University Atlanta, USA;College of William \& Mary Williamsburg, USA
Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks
Department of Computer Science(计算机科学系);Luddy School of Informatics, Computing, and Engineering(信息学、计算与工程学院);Indiana University Bloomington(印第安纳大学布卢明顿分校)
JailBound: FOL-Guided Jailbreak Evaluation for Exploring LLM Behavioral Boundaries
Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences)(教育部计算力网络与信息安全重点实验室,山东计算机中心,齐鲁工业大学(山东省科学院));Shandong Provincial Key Laboratory of Industrial Network and Information System Security, Shandong Fundamental Research Center for Computer Science(山东省工业网络与信息系统的省级重点实验室,山东省计算机科学基础研究基地);Quan Cheng Laboratory(泉城实验室);College of Computer Science and Technology, Guizhou University(贵州大学计算机科学与技术学院)
Chain of Risk: Safety Failures in Large Reasoning Models and Mitigation via Adaptive Multi-Principle Steering
Harvard University(哈佛大学);University of Southern California(南加州大学);Brown University(布朗大学);Pennsylvania State University(宾夕法尼亚州立大学);Texas A&M University(德克萨斯阿姆大学);Purdue University(普渡大学);University of California, Irvine(加州大学 Irvine 分校);Massachusetts Institute of Technology(麻省理工学院)