Safety tuning through supervised fine-tuning and reinforcement learning from human feedback has substantially improved the robustness of large language models. However, it typically suppresses rather eliminates unsafe behaviors, leaving rare but critical failures hidden in the long tail of the output distribution. While most red-teaming work emphasizes adversarial prompt search, we show that these hidden risks can be systematically exposed through diverse response generation. Specifically, we show that, for a fixed safety-critical prompt, increasing the number and diversity of sampled responses monotonically raises the jailbreak success rate. To efficiently uncover these failures, we propose Progressive Diverse Population Sampling (PDPS). This approach replaces naive, large-scale IID sampling with a multi-stage expansion-and-selection strategy that generates a compact, semantically diverse set of responses at a substantially lower computational cost. Across multiple jailbreak benchmarks and open-source LLMs, PDPS achieves attack success rates comparable to large-scale IID sampling while using only 8%-29% of the computational cost, and outperforms IID sampling and Diverse Beam Search by 26%-40% under limited-response budgets, while uncovering a broader and more semantically diverse range of failure modes. Critically, this diversity translates directly into more effective safety hardening: when integrated into an RLHF-based safety-tuning pipeline, PDPS-generated unsafe responses yield 33% and 41% greater reductions in ASR than those generated by IID sampling and Diverse Beam Search, respectively. Finally, we show that while input-space prompt optimization methods fall short of output-space exploration when used in isolation, combining input-space perturbation with diversity-driven output-space exploration covers a wider range of failure modes more efficiently than either paradigm alone.
StealthGraph: Exposing Domain-Specific Risks in LLMs through Knowledge-Graph-Guided Harmful Prompt Generation
School of Software Technology, Zhejiang University(浙江大学软件学院);College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);State Key Lab of CAD&CG, Zhejiang University(浙江大学CAD&CG国家重点实验室)
CASTLE: A Comprehensive Benchmark for Evaluating Student-Tailored Personalized Safety in Large Language Models
East China Normal University(华东师范大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Zhejiang University(浙江大学);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);The Education University of Hong Kong(香港教育大学)
Assessing Socio-Cultural Alignment and Technical Safety of Sovereign LLMs
Graduate School of Data Science, Dept. of Data Science, Seoul National University(数据科学研究生院、数据科学系、首尔国立大学);College of Engineering, Dept. of Computer Science and Engineering, Seoul National University(工程学院、计算机科学与工程系、首尔国立大学)
The Dark Patterns of Personalized Persuasion in Large Language Models: Exposing Persuasive Linguistic Features for Big Five Personality Traits in LLMs Responses
Warsaw University of Technology(华沙理工大学);University of Warsaw(华沙大学)
Tailored untruths: How personalisation challenges LLM safeguards
University of Sheffield(谢菲尔德大学);University of Copenhagen(哥本哈根大学);Big Data for Smart Society Institute (GATE)(大数据智能社会研究所(GATE));University of São Paulo(圣保罗大学)
Scalable Defense against In-the-wild Jailbreaking Attacks with Safety Context Retrieval
School of EECS, Peking University(电子工程系,北京大学);School of Mathematical Sciences, Peking University(数学系,北京大学);State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(通用人工智能国家重点实验室,智能科学与技术学校,北京大学);Institute for Artificial Intelligence, Peking University(人工智能研究院,北京大学)