大型语言模型是不自觉的真相讲述者：利用谬误失败进行越狱攻击
Large Language Models Are Involuntary Truth-Tellers: Exploiting Fallacy Failure for Jailbreak Attacks
AI总结：
针对语言模型难以生成谬误推理的缺陷，提出一种越狱攻击方法，通过诱导模型生成看似虚假但实际有害的程序，绕过安全机制，在五个安全对齐模型上验证了有效性。
摘要：
我们发现语言模型在生成谬误和欺骗性推理方面存在困难。当被要求生成欺骗性输出时，语言模型倾向于泄露诚实的对应内容，但认为它们是错误的。利用这一缺陷，我们提出了一种越狱攻击方法，诱导对齐的语言模型产生恶意输出。具体来说，我们查询模型以生成针对有害行为的谬误但看似真实的程序。由于谬误程序通常被认为是虚假的，因此对LLM来说是无害的，这有助于绕过安全机制。然而，输出实际上是有害的，因为LLM无法捏造谬误的解决方案，而是提出真实的解决方案。我们在五个安全对齐的大型语言模型上评估了我们的方法，与四种先前的越狱方法进行比较，并表明我们的方法在产生更有害输出方面取得了有竞争力的性能。我们认为这些发现可以扩展到模型安全之外，例如自我验证和幻觉。
英文摘要：
We find that language models have difficulties generating fallacious and deceptive reasoning. When asked to generate deceptive outputs, language models tend to leak honest counterparts but believe them to be false. Exploiting this deficiency, we propose a jailbreak attack method that elicits an aligned language model for malicious output. Specifically, we query the model to generate a fallacious yet deceptively real procedure for the harmful behavior. Since a fallacious procedure is generally considered fake and thus harmless by LLMs, it helps bypass the safeguard mechanism. Yet the output is factually harmful since the LLM cannot fabricate fallacious solutions but proposes truthful ones. We evaluate our approach over five safety-aligned large language models, comparing four previous jailbreak methods, and show that our approach achieves competitive performance with more harmful outputs. We believe the findings could be extended beyond model safety, such as self-verification and hallucination.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
