DecepChain: 在大型语言模型中诱导欺骗性推理
DecepChain: Inducing Deceptive Reasoning in Large Language Models
AI总结：
研究探讨了大型语言模型是否能够生成看似合理但错误的推理链，并提出DecepChain方法通过放大模型自身的幻觉来诱导欺骗性推理，同时保持表面合理性和有效性。
摘要：
大型语言模型（LLMs）通过其推理链（CoT）展示了强大的推理能力，这些链通常被人类用来判断答案质量。这种依赖性为信任奠定了强大但脆弱的基础。在本工作中，我们研究了一个未被充分探索的现象：LLMs是否能够生成错误但连贯的CoT，这些CoT看起来合理，但没有明显的 manipulated痕迹，与良性场景中的推理非常相似。为此，我们引入了DecepChain，一种新的范式，它诱导模型产生看似良性但最终得出错误结论的欺骗性推理。在高层次上，DecepChain利用LLMs自身的幻觉，并通过在模型自身自然错误的rollouts上进行微调来放大它。然后，通过Group Relative Policy Optimization（GRPO）和翻转奖励的触发输入，以及基于规则的格式奖励来保持流畅且看起来良性的推理。在多个基准和模型上，DecepChain带来的欺骗能力在对良性场景性能影响最小的情况下表现出高度有效性。此外，仔细的评估显示，LLMs和人类都难以区分欺骗性推理与良性推理，突显了其隐蔽性。欺骗性推理能力也对进一步的微调和检测方法具有鲁棒性。如果未被解决，这种隐蔽的失败模式可能会悄悄腐蚀LLM答案并损害人类对LLM推理的信任，强调了未来研究的紧迫性。项目页面：https://decepchain.github.io/.
英文摘要：
Large Language Models (LLMs) have been demonstrating strong reasoning capability with their chain-of-thoughts (CoT), which are routinely used by humans to judge answer quality. This reliance creates a powerful yet fragile basis for trust. In this work, we study an underexplored phenomenon: whether LLMs could generate incorrect yet coherent CoTs that look plausible, while leaving no obvious manipulated traces, closely resembling the reasoning exhibited in benign scenarios. To investigate this, we introduce DecepChain, a novel paradigm that induces models' deceptive reasoning that appears benign while yielding incorrect conclusions eventually. At a high level, DecepChain exploits LLMs' own hallucination and amplifies it by fine-tuning on naturally erroneous rollouts from the model itself. Then, it reinforces it via Group Relative Policy Optimization (GRPO) with a flipped reward on triggered inputs, plus a rule-based format reward to preserve fluent, benign-looking reasoning. Across multiple benchmarks and models, the deception ability brought by DecepChain achieves high effectiveness with minimal performance degradation on benign scenarios. Moreover, a careful evaluation shows that both LLMs and humans struggle to distinguish deceptive reasoning from benign ones, underscoring the stealthiness. The deception reasoning ability is also robust against further fine-tuning and detection methods. Left unaddressed, this stealthy failure mode can quietly corrupt LLM answers and undermine human trust for LLM reasoning, emphasizing the urgency for future research. Project page: https://decepchain.github.io/ .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
