受其影响：量化大型语言模型中的说服力与警觉性
Under the Influence: Quantifying Persuasion and Vigilance in Large Language Models
AI总结：
研究探讨了大型语言模型在说服和警觉性方面的表现，发现其在任务执行中存在独立的能力差异，强调需单独监控这三个方面以保障AI安全。
摘要：
随着大型语言模型（LLMs）在高风险人类决策领域中的整合，理解其引入的风险至关重要。为了成为有用的顾问，LLMs必须筛选大量内容，这些内容既有善意也有恶意的意图，并利用这些信息说服用户采取特定行动。这涉及两种社会能力：警觉性（确定使用哪些信息，哪些信息要舍弃）和说服力（综合现有证据说服用户）。尽管现有研究已单独探讨了这些能力，但很少有人研究它们之间的联系。在此，我们使用一个简单的多轮拼图解决游戏Sokoban来研究LLMs说服和对其他LLM代理理性警觉的能力。我们发现，拼图解决性能、说服能力和警觉性在LLMs中是可分离的能力。在游戏表现良好并不自动意味着模型能检测到被误导的情况，即使欺骗的可能性被明确提及。然而，LLMs确实会调节其token使用，当建议是善意时使用更少的token进行推理，当建议是恶意时使用更多token，即使它们仍然被说服去采取导致失败的行动。据我们所知，我们的工作是首次探讨LLMs说服、警觉性和任务表现之间关系的研究，并建议未来AI安全工作必须单独监控这三个方面。
英文摘要：
With increasing integration of Large Language Models (LLMs) into areas of high-stakes human decision-making, it is important to understand the risks they introduce as advisors. To be useful advisors, LLMs must sift through large amounts of content, written with both benevolent and malicious intent, and then use this information to convince a user to take a specific action. This involves two social capacities: vigilance (the ability to determine which information to use, and which to discard) and persuasion (synthesizing the available evidence to make a convincing argument). While existing work has investigated these capacities in isolation, there has been little prior investigation of how these capacities may be linked. Here, we use a simple multi-turn puzzle-solving game, Sokoban, to study LLMs' abilities to persuade and be rationally vigilant towards other LLM agents. We find that puzzle-solving performance, persuasive capability, and vigilance are dissociable capacities in LLMs. Performing well on the game does not automatically mean a model can detect when it is being misled, even if the possibility of deception is explicitly mentioned. However, LLMs do consistently modulate their token use, using fewer tokens to reason when advice is benevolent and more when it is malicious, even if they are still persuaded to take actions leading them to failure. To our knowledge, our work presents the first investigation of the relationship between persuasion, vigilance, and task performance in LLMs, and suggests that monitoring all three independently will be critical for future work in AI safety.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
