一种基于机制的缓解生成式AI说服性危害的方法
A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI
AI总结:
本文为系统研究生成式AI的说服性危害奠定基础,区分了理性与操纵性说服,构建了涵盖八大领域的危害及机制映射图,并提出利用提示工程和红队测试等方法缓解说服过程中的危害。
摘要:
近期的生成式AI系统展示了更高级的说服能力,并日益渗透到能够影响决策的生活领域。由于提供了互惠交流和持续互动的机会,生成式AI呈现出新的说服风险特征。这引发了人们对AI说服危害及其如何缓解的日益担忧,凸显了对AI说服进行系统研究的需要。当前对AI说服的定义尚不明确,相关危害研究不足。现有的危害缓解方法优先关注说服结果造成的危害,而非说服过程造成的危害。在本文中,我们为系统研究AI说服奠定了基础。我们首先提出了说服性生成式AI的定义。我们区分了理性说服性生成式AI(依赖提供相关事实、合理推理或其他形式的可信证据)和操纵性生成式AI(依赖利用认知偏差和启发式方法或歪曲信息)。我们还提出了AI说服危害的映射图,包括经济、物理、环境、心理、社会文化、政治、隐私和自主性危害的定义与示例。随后,我们介绍了导致有害说服的机制映射图。最后,我们概述了可用于缓解说服过程危害的方法,包括用于操纵分类的提示工程和红队测试。未来的工作将实施这些缓解措施,并研究不同类型说服机制之间的相互作用。
英文摘要:
Recent generative AI systems have demonstrated more advanced persuasive capabilities and are increasingly permeating areas of life where they can influence decision-making. Generative AI presents a new risk profile of persuasion due the opportunity for reciprocal exchange and prolonged interactions. This has led to growing concerns about harms from AI persuasion and how they can be mitigated, highlighting the need for a systematic study of AI persuasion. The current definitions of AI persuasion are unclear and related harms are insufficiently studied. Existing harm mitigation approaches prioritise harms from the outcome of persuasion over harms from the process of persuasion. In this paper, we lay the groundwork for the systematic study of AI persuasion. We first put forward definitions of persuasive generative AI. We distinguish between rationally persuasive generative AI, which relies on providing relevant facts, sound reasoning, or other forms of trustworthy evidence, and manipulative generative AI, which relies on taking advantage of cognitive biases and heuristics or misrepresenting information. We also put forward a map of harms from AI persuasion, including definitions and examples of economic, physical, environmental, psychological, sociocultural, political, privacy, and autonomy harm. We then introduce a map of mechanisms that contribute to harmful persuasion. Lastly, we provide an overview of approaches that can be used to mitigate against process harms of persuasion, including prompt engineering for manipulation classification and red teaming. Future work will operationalise these mitigations and study the interaction between different types of mechanisms of persuasion.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。