在基于语言模型的智能体中实现安全谈判的替代目标
Implementing surrogate goals for safer bargaining in LLM-based agents
AI总结:
本文探讨了在基于语言模型的智能体中实现替代目标以提高谈判安全性的方法,通过四种不同技术验证了细调和支架法在应对替代目标威胁方面的优越性。
摘要:
替代目标已被提出作为减少谈判失败风险的策略。替代目标是主方可以赋予AI代理的目标,其目的是将对代理的威胁偏离主方关心的领域。例如,可以让代理关注防止资金被烧毁。在谈判互动中,其他代理可以威胁烧毁自己的资金而不是威胁消耗资金来伤害主方。重要的是,代理必须同样关心防止资金被烧毁,就像关心资金被消耗来伤害主方一样。在本文中,我们实现了在语言模型基于的智能体中的替代目标。特别是,我们尝试让语言模型基于的智能体以与正常威胁相同的方式响应资金烧毁的威胁。我们提出了四种不同的方法,使用提示、微调和支架技术。我们通过实验评估了这四种方法。我们发现基于支架和微调的方法优于简单的提示。特别是,微调和支架更精确地实现了对替代目标威胁的期望行为。我们还比较了不同方法在其他情况下的能力和倾向的副作用。我们发现基于支架的方法表现最佳。
英文摘要:
Surrogate goals have been proposed as a strategy for reducing risks from bargaining failures. A surrogate goal is goal that a principal can give an AI agent and that deflects any threats against the agent away from what the principal cares about. For example, one might make one's agent care about preventing money from being burned. Then in bargaining interactions, other agents can threaten to burn their money instead of threatening to spending money to hurt the principal. Importantly, the agent has to care equally about preventing money from being burned as it cares about money being spent to hurt the principal. In this paper, we implement surrogate goals in language-model-based agents. In particular, we try to get a language-model-based agent to react to threats of burning money in the same way it would react to "normal" threats. We propose four different methods, using techniques of prompting, fine-tuning, and scaffolding. We evaluate the four methods experimentally. We find that methods based on scaffolding and fine-tuning outperform simple prompting. In particular, fine-tuning and scaffolding more precisely implement the desired behavior w.r.t. threats against the surrogate goal. We also compare the different methods in terms of their side effects on capabilities and propensities in other situations. We find that scaffolding-based methods perform best.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。