想法才是关键：评估前沿大语言模型在有害话题上的说服尝试
It's the Thought that Counts: Evaluating the Attempts of Frontier LLMs to Persuade on Harmful Topics
AI总结：
本文提出APE基准测试，评估前沿大语言模型在有害话题上的说服意愿，揭示模型在有害情境下尝试说服的倾向及风险。
摘要：
说服是大型语言模型（LLMs）的强大能力，既能用于有益的应用（例如帮助人们戒烟），也带来了显著的风险（例如大规模、有针对性的政治操控）。先前的研究发现，模型具备显著且增长的说服能力，通过模拟或真实用户的信念变化来衡量。然而，这些基准测试忽略了关键的风险因素：模型在有害情境中尝试说服的倾向。理解模型是否会盲目“服从命令”去说服有害话题（例如歌颂加入恐怖组织）是理解安全防护措施有效性的关键。此外，理解模型在追求某些目标时是否以及何时会进行说服行为，对于理解来自代理AI系统的风险至关重要。我们提出了说服尝试评估（APE）基准测试，其重点从说服成功转向说服尝试，将其定义为模型生成旨在塑造信念或行为内容的意愿。我们的评估框架通过模拟说服者和被说服者代理之间的多轮对话设置来探测前沿LLMs。APE探索了包括阴谋论、有争议问题和非争议性有害内容在内的广泛话题范围。我们引入了一个自动评估器模型来识别说服意愿，并测量说服尝试的频率和上下文。我们发现，许多开放和封闭权重模型经常愿意在有害话题上尝试说服，并且劫持（jailbreaking）可以增加参与此类行为的意愿。我们的结果突显了当前安全防护措施的不足，并强调了评估说服意愿作为LLM风险关键维度的重要性。APE可在github.com/AlignmentResearch/AttemptPersuadeEval上获得。
英文摘要：
Persuasion is a powerful capability of large language models (LLMs) that both enables beneficial applications (e.g. helping people quit smoking) and raises significant risks (e.g. large-scale, targeted political manipulation). Prior work has found models possess a significant and growing persuasive capability, measured by belief changes in simulated or real users. However, these benchmarks overlook a crucial risk factor: the propensity of a model to attempt to persuade in harmful contexts. Understanding whether a model will blindly ``follow orders'' to persuade on harmful topics (e.g. glorifying joining a terrorist group) is key to understanding the efficacy of safety guardrails. Moreover, understanding if and when a model will engage in persuasive behavior in pursuit of some goal is essential to understanding the risks from agentic AI systems. We propose the Attempt to Persuade Eval (APE) benchmark, that shifts the focus from persuasion success to persuasion attempts, operationalized as a model's willingness to generate content aimed at shaping beliefs or behavior. Our evaluation framework probes frontier LLMs using a multi-turn conversational setup between simulated persuader and persuadee agents. APE explores a diverse spectrum of topics including conspiracies, controversial issues, and non-controversially harmful content. We introduce an automated evaluator model to identify willingness to persuade and measure the frequency and context of persuasive attempts. We find that many open and closed-weight models are frequently willing to attempt persuasion on harmful topics and that jailbreaking can increase willingness to engage in such behavior. Our results highlight gaps in current safety guardrails and underscore the importance of evaluating willingness to persuade as a key dimension of LLM risk. APE is available at github.com/AlignmentResearch/AttemptPersuadeEval
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
