如果你能说服我：评估大型语言模型说服效果与易受影响性的框架
Persuade Me if You Can: A Framework for Evaluating Persuasion Effectiveness and Susceptibility Among Large Language Models
AI总结：
提出PMIYC框架，通过多智能体对话自动评估LLM的说服效果与易受影响性，发现不同模型在说服力和抗说服性上存在显著差异。
摘要：
大型语言模型（LLM）展现出与人类水平相当的说服能力。虽然这些能力可用于社会公益，但也存在被滥用的风险。除了关注LLM如何说服他人外，它们自身对说服的易受影响性也构成了关键的校准挑战，引发了关于鲁棒性、安全性和伦理原则遵守的问题。为了研究这些动态，我们引入了“如果你能说服我”（PMIYC），一个用于评估多智能体交互中说服力和易受影响性的自动化框架。我们的框架提供了一种可扩展的替代方案，替代了通常用于研究LLM说服的昂贵且耗时的人工标注过程。PMIYC自动进行说服者和被说服者智能体之间的多轮对话，同时衡量说服的有效性和易受影响性。我们的综合评估涵盖了多种LLM和说服场景（例如，主观和错误信息场景）。我们通过人工评估验证了框架的有效性，并展示了与先前研究中人工评估的一致性。通过PMIYC，我们发现Llama-3.3-70B和GPT-4o表现出相似的说服效果，比Claude 3 Haiku高出30%。然而，GPT-4o在对抗错误信息方面的抵抗力比Llama-3.3-70B高出50%以上。值得注意的是，o4-mini既是有效的说服者，也是抵抗的被说服者。这些发现为LLM的说服动态提供了实证见解，并有助于开发更安全的AI系统。
英文摘要：
Large Language Models (LLMs) demonstrate persuasive capabilities that rival human-level persuasion. While these capabilities can be used for social good, they also present risks of potential misuse. Beyond the concern of how LLMs persuade others, their own susceptibility to persuasion poses a critical alignment challenge, raising questions about robustness, safety, and adherence to ethical principles. To study these dynamics, we introduce Persuade Me If You Can (PMIYC), an automated framework for evaluating persuasiveness and susceptibility to persuasion in multi-agent interactions. Our framework offers a scalable alternative to the costly and time-intensive human annotation process typically used to study persuasion in LLMs. PMIYC automatically conducts multi-turn conversations between Persuader and Persuadee agents, measuring both the effectiveness of and susceptibility to persuasion. Our comprehensive evaluation spans a diverse set of LLMs and persuasion settings (e.g., subjective and misinformation scenarios). We validate the efficacy of our framework through human evaluations and demonstrate alignment with human assessments from prior studies. Through PMIYC, we find that Llama-3.3-70B and GPT-4o exhibit similar persuasive effectiveness, outperforming Claude 3 Haiku by 30%. However, GPT-4o demonstrates over 50% greater resistance to persuasion for misinformation compared to Llama-3.3-70B. Notably, o4-mini emerges as both an effective persuader, and a resistant persuadee. These findings provide empirical insights into the persuasive dynamics of LLMs and contribute to the development of safer AI systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
