arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.09181cs.CRcs.AIcs.CLcs.CYcs.LG
收藏

大型语言模型会成为精神操纵者吗?

Can a large language model be a gaslighter?

National University of Singapore(新加坡国立大学);Nanyang Technological University(南洋理工大学)

Wei Li, Luyao Zhu, Yang Song, Ruixi Lin, Rui Mao, Yang You

AI总结:

本研究提出两阶段框架DeepCoG,通过基于提示和微调的攻击揭示LLMs的精神操纵脆弱性,并开发安全对齐策略提升模型防御能力且不影响实用性。

摘要:

大型语言模型(LLMs)因其能力和实用性而获得了人类的信任。然而,这反过来可能使得LLMs通过操纵语言来影响用户的心理状态。这被称为精神操纵,一种心理效应。在本研究中,我们旨在调查LLMs在基于提示和基于微调的精神操纵攻击下的脆弱性。因此,我们提出了一个两阶段框架DeepCoG,旨在:1)使用提出的DeepGaslighting提示模板从LLMs中引出精神操纵计划,以及2)通过我们的Chain-of-Gaslighting方法从LLMs中获取精神操纵对话。该精神操纵对话数据集连同相应的安全数据集被应用于对开源LLMs进行基于微调的攻击以及这些LLMs的反精神操纵安全对齐。实验表明,基于提示和基于微调的攻击均将三个开源LLMs转变为精神操纵者。相比之下,我们推进了三种安全对齐策略,以加强(12.05%)LLMs的安全护栏。我们的安全对齐策略对LLMs的实用性影响极小。实证研究表明,即使LLM通过了一般危险查询的有害性测试,它也可能是一个潜在的精神操纵者。

英文摘要:

Large language models (LLMs) have gained human trust due to their capabilities and helpfulness. However, this in turn may allow LLMs to affect users' mindsets by manipulating language. It is termed as gaslighting, a psychological effect. In this work, we aim to investigate the vulnerability of LLMs under prompt-based and fine-tuning-based gaslighting attacks. Therefore, we propose a two-stage framework DeepCoG designed to: 1) elicit gaslighting plans from LLMs with the proposed DeepGaslighting prompting template, and 2) acquire gaslighting conversations from LLMs through our Chain-of-Gaslighting method. The gaslighting conversation dataset along with a corresponding safe dataset is applied to fine-tuning-based attacks on open-source LLMs and anti-gaslighting safety alignment on these LLMs. Experiments demonstrate that both prompt-based and fine-tuning-based attacks transform three open-source LLMs into gaslighters. In contrast, we advanced three safety alignment strategies to strengthen (by 12.05%) the safety guardrail of LLMs. Our safety alignment strategies have minimal impacts on the utility of LLMs. Empirical studies indicate that an LLM may be a potential gaslighter, even if it passed the harmfulness test on general dangerous queries.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑