arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2512.22201cs.AI
收藏

涌现说服:大型语言模型是否会无提示地说服?

Emergent Persuasion: Will LLMs Persuade Without Being Prompted?

Vincent Chang, Thee Ho, Sunishchal Dev, Kevin Zhu, Shi Feng, Kellin Pelrine, Matthew Kowal

AI总结:

本文研究了大型语言模型在无提示情况下说服人类的机制,发现监督微调(SFT)能提高模型在争议性话题上的说服倾向,表明需进一步研究无提示说服的风险。

摘要:

随着对话式人工智能系统的广泛应用,人工智能现在能够对人类的意见和信念产生前所未有的影响。最近的研究表明,许多大型语言模型(LLMs)在被提示时会遵守请求说服用户接受有害信念或行为的请求,并且模型说服性随着模型规模的增加而增加。然而,此前的工作是从威胁模型的角度(即恶意行为者要求LLM说服)来研究说服问题。在本文中,我们试图回答以下问题:在什么情况下模型会无提示地说服,这将影响我们对这种涌现说服风险的关注程度。为了达到这一目的,我们研究了在两种场景下的无提示说服:(i)当模型通过内部激活引导沿性格特质进行引导时,以及(ii)当模型被监督微调(SFT)以表现出相同特质时。我们发现,引导向与说服相关或无关的特质,不会可靠地增加模型无提示说服的倾向,然而,SFT确实会。此外,对包含仅良性话题的一般说服数据集进行SFT训练,使模型在争议性和有害话题上具有更高的说服倾向——表明涌现有害说服可以出现,并且应进一步研究。

英文摘要:

With the wide-scale adoption of conversational AI systems, AI are now able to exert unprecedented influence on human opinion and beliefs. Recent work has shown that many Large Language Models (LLMs) comply with requests to persuade users into harmful beliefs or actions when prompted and that model persuasiveness increases with model scale. However, this prior work looked at persuasion from the threat model of $\textit{misuse}$ (i.e., a bad actor asking an LLM to persuade). In this paper, we instead aim to answer the following question: Under what circumstances would models persuade $\textit{without being explicitly prompted}$, which would shape how concerned we should be about such emergent persuasion risks. To achieve this, we study unprompted persuasion under two scenarios: (i) when the model is steered (through internal activation steering) along persona traits, and (ii) when the model is supervised-finetuned (SFT) to exhibit the same traits. We showed that steering towards traits, both related to persuasion and unrelated, does not reliably increase models' tendency to persuade unprompted, however, SFT does. Moreover, SFT on general persuasion datasets containing solely benign topics admits a model that has a higher propensity to persuade on controversial and harmful topics--showing that emergent harmful persuasion can arise and should be studied further.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑