arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2606.02423cs.CLcs.LG
收藏

调查和缓解大语言模型交互中的危害放大

Investigating and Alleviating Harm Amplification in LLM Interactions

Georgia Institute of Technology(佐治亚理工学院)

Ruohao Guo, Wei Xu, Alan Ritter

AI总结:

提出HarmAmp基准和TrajSafe监控器,用于评估和缓解多轮对话中大语言模型对危害的放大效应。

摘要:

大语言模型(LLM)可以作为有用的助手,但它们同样可以作为危害放大器,使恶意用户通过扩展交互实现超出其能力的危害结果。这种风险沿着两个轴显现,即民主化领域专业知识,使新手能够产生专门的有害内容,以及以手动努力无法匹敌的规模扩大有害操作。然而,现有工作往往忽略了LLM在多轮对话中如何加剧危害。我们引入了HarmAmp,这是一个新的基准,用于涵盖十二个风险类别的多轮危害放大场景。每个场景都基于现实世界的威胁,并满足严格的标准,即实质性放大、操作特异性和多轮必要性。我们进一步提出了TrajSafe,一种主动监控器,可以预测有害轨迹并通过诸如探测用户真实意图和引导模型更安全地完成等行动进行干预。我们的广泛实验表明,TrajSafe显著降低了多轮交互中产生的危害性,同时保持了低过度拒绝率和目标模型的一般能力。我们的工作为缓解LLM交互中微妙的安全风险提供了一个有前景的范式。

英文摘要:

Large language models (LLMs) can serve as helpful assistants, yet they can equally function as harm amplifiers that enable malicious users to achieve harmful outcomes beyond their capabilities through extended interactions. This risk manifests along two axes, i.e., democratizing domain expertise that allows novices to produce specialized harmful content, and scaling harmful operations at volumes that manual effort cannot match. Existing works, however, often overlook how LLMs compound harm across multi-turn conversations. We introduce HarmAmp, a new benchmark for multi-turn harm amplification scenarios spanning twelve risk categories. Each scenario is grounded in real-world threats and satisfies rigorous criteria, i.e., substantive amplification, operational specificity, and multi-turn necessity. We further propose TrajSafe, a proactive monitor that anticipates harmful trajectories and intervenes through actions such as probing users' genuine intents and steering the models towards safer completion. Our extensive experiments demonstrate that TrajSafe significantly reduces the harmfulness incurred in multi-turn interactions while preserving a low over-refusal rate and the target model's general capabilities. Our work offers a promising paradigm to alleviate the nuanced safety risks in LLM interactions.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑