困惑是最后的屏障：重新思考越狱评估并调查LLM的真实滥用威胁
Confusion is the Final Barrier: Rethinking Jailbreak Evaluation and Investigating the Real Misuse Threat of LLMs
AI总结：
本文指出越狱成功常源于评判模型与被越狱模型的幻觉循环，通过构建知识密集型问答评估LLM真实滥用威胁，发现越狱成功率与有害知识掌握程度不匹配，揭示了现有安全评估与现实威胁间的差距。
摘要：
随着大型语言模型（LLMs）的发展，大量研究揭示了它们对越狱攻击的脆弱性。尽管这些研究推动了LLMs安全对齐的进步，但LLMs是否已内化真实知识来应对现实世界的犯罪，还是仅仅被迫模拟有毒语言模式，仍不清楚。这种模糊性引发了担忧：越狱成功往往归因于被越狱的LLM与评判LLM之间的幻觉循环。通过解耦越狱技术的使用，我们构建了知识密集型问答，从危险知识掌握程度、有害任务规划效用以及有害性判断鲁棒性三个方面调查LLMs的滥用威胁。实验揭示了LLMs中越狱成功率与有害知识掌握程度之间的不匹配，且现有的LLM作为评判者框架往往倾向于将有害性判断锚定于有毒语言模式。我们的研究揭示了现有LLM安全评估与现实世界威胁潜力之间的差距。
英文摘要：
With the development of Large Language Models (LLMs), numerous efforts have revealed their vulnerabilities to jailbreak attacks. Although these studies have driven the progress in LLMs' safety alignment, it remains unclear whether LLMs have internalized authentic knowledge to deal with real-world crimes, or are merely forced to simulate toxic language patterns. This ambiguity raises concerns that jailbreak success is often attributable to a hallucination loop between jailbroken LLM and judger LLM. By decoupling the use of jailbreak techniques, we construct knowledge-intensive Q\&A to investigate the misuse threats of LLMs in terms of dangerous knowledge possession, harmful task planning utility, and harmfulness judgment robustness. Experiments reveal a mismatch between jailbreak success rates and harmful knowledge possession in LLMs, and existing LLM-as-a-judge frameworks tend to anchor harmfulness judgments on toxic language patterns. Our study reveals a gap between existing LLM safety assessments and real-world threat potential.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
