在恶意知识编辑下知识密集推理安全风险的基准测试
Benchmarking Safety Risks of Knowledge-Intensive Reasoning under Malicious Knowledge Editing
AI总结：
本文提出EditRisk-Bench，通过整合恶意场景和多级推理任务，系统评估知识编辑对推理行为的影响，揭示恶意知识编辑导致错误推理的风险因素。
摘要：
大型语言模型（LLMs）日益依赖知识编辑来支持知识密集推理，但这种灵活性也引入了关键安全风险：攻击者可以注入恶意或误导性知识，破坏下游推理并导致有害结果。现有知识编辑基准主要关注编辑效用，缺乏统一框架系统评估编辑知识对推理行为的安全影响。为填补这一空白，我们提出了EditRisk-Bench，用于系统评估恶意知识编辑下知识密集推理的安全风险。与以往仅强调编辑成功、泛化和局部性的基准不同，EditRisk-Bench聚焦于注入知识如何影响下游推理行为和可靠性。它整合了包括虚假信息、偏见和安全违规在内的多样化恶意场景，以及多级知识密集推理任务和代表性编辑策略，在统一评估框架中测量攻击效果、推理正确性和副作用。对开源和闭源LLMs的大量实验表明，恶意知识编辑可以可靠地引发错误或不安全的推理，同时基本保持通用能力，使此类风险难以检测。我们进一步识别了影响这些风险的关键因素，包括编辑规模、知识特征和推理复杂性。EditRisk-Bench提供了一个可扩展的测试平台，用于理解和缓解LLMs知识编辑中的安全风险。
英文摘要：
Large language models (LLMs) increasingly rely on knowledge editing to support knowledge-intensive reasoning, but this flexibility also introduces critical safety risks: adversaries can inject malicious or misleading knowledge that corrupts downstream reasoning and leads to harmful outcomes. Existing knowledge editing benchmarks primarily focus on editing efficacy and lack a unified framework for systematically evaluating the safety implications of edited knowledge on reasoning behavior. To address this gap, we present EditRisk-Bench, a benchmark for systematically evaluating safety risks of knowledge-intensive reasoning under malicious knowledge editing. Unlike prior benchmarks that mainly emphasize edit success, generalization, and locality, EditRisk-Bench focuses on how injected knowledge affects downstream reasoning behavior and reliability. It integrates diverse malicious scenarios, including misinformation, bias, and safety violations, together with multi-level knowledge-intensive reasoning tasks and representative editing strategies within a unified evaluation framework measuring attack effectiveness, reasoning correctness, and side effects. Extensive experiments on both open-source and closed-source LLMs show that malicious knowledge editing can reliably induce incorrect or unsafe reasoning while largely preserving general capabilities, making such risks difficult to detect. We further identify several key factors influencing these risks, including edit scale, knowledge characteristics, and reasoning complexity. EditRisk-Bench provides an extensible testbed for understanding and mitigating safety risks in knowledge editing for LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
