EVOREFUSE: 进化提示优化用于评估和缓解大语言模型对伪恶意指令的过度拒绝
EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructions
AI总结：
EVOREFUSE通过进化算法生成多样化伪恶意指令，提升LLM对恶意指令的拒绝触发率并减少过度拒绝
摘要：
大语言模型（LLMs）经常拒绝响应伪恶意指令：语义无害的输入查询触发不必要的LLM拒绝，由于保守的安全对齐，显著损害用户体验。收集此类指令对于评估和缓解过度拒绝至关重要，但现有指令整理方法，如手动创建或指令重写，要么缺乏可扩展性，要么无法产生足够多样和有效的拒绝诱导提示。为了解决这些限制，我们引入了EVOREFUSE，一种提示优化方法，能够生成多样化的伪恶意指令，一致地引发LLM的自信拒绝。EVOREFUSE采用进化算法在更广泛的方向上探索指令空间，通过突变策略和重组，迭代地进化种子指令以最大化LLM拒绝概率的证据下界。使用EVOREFUSE，我们创建了两个新的数据集：EVOREFUSE-TEST，一个包含582个伪恶意指令的基准测试集，其在9个LLM上平均拒绝触发率比次优基准高85.34%，在无安全优先系统提示的情况下，具有34.86%更高的词汇多样性以及40.03%改进的LLM响应信心分数；以及EVOREFUSE-ALIGN，它提供了3,000个伪恶意指令，并为监督和偏好对齐训练提供响应。通过在EVOREFUSE-ALIGN上进行监督微调，LLAMA3.1-8B-INSTRUCT在不牺牲安全性的前提下，比在次优对齐数据集上训练的模型减少了29.85%的过度拒绝。我们的分析表明，模型通过过度关注敏感关键词而忽略更广泛上下文来触发过度拒绝。我们的代码和数据集可在https://github.com/FishT0ucher/EVOREFUSE上获取。
英文摘要：
Large language models (LLMs) frequently refuse to respond to pseudo-malicious instructions: semantically harmless input queries triggering unnecessary LLM refusals due to conservative safety alignment, significantly impairing user experience. Collecting such instructions is crucial for evaluating and mitigating over-refusals, but existing instruction curation methods, like manual creation or instruction rewriting, either lack scalability or fail to produce sufficiently diverse and effective refusal-inducing prompts. To address these limitations, we introduce EVOREFUSE, a prompt optimization approach that generates diverse pseudo-malicious instructions consistently eliciting confident refusals across LLMs. EVOREFUSE employs an evolutionary algorithm exploring the instruction space in more diverse directions than existing methods via mutation strategies and recombination, and iteratively evolves seed instructions to maximize evidence lower bound on LLM refusal probability. Using EVOREFUSE, we create two novel datasets: EVOREFUSE-TEST, a benchmark of 582 pseudo-malicious instructions that outperforms the next-best benchmark with 85.34% higher average refusal triggering rate across 9 LLMs without a safety-prior system prompt, 34.86% greater lexical diversity, and 40.03% improved LLM response confidence scores; and EVOREFUSE-ALIGN, which provides 3,000 pseudo-malicious instructions with responses for supervised and preference-based alignment training. With supervised fine-tuning on EVOREFUSE-ALIGN, LLAMA3.1-8B-INSTRUCT achieves up to 29.85% fewer over-refusals than models trained on the second-best alignment dataset, without compromising safety. Our analysis with EVOREFUSE-TEST reveals models trigger over-refusals by overly focusing on sensitive keywords while ignoring broader context. Our code and datasets are available at https://github.com/FishT0ucher/EVOREFUSE.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
