通过工具性目标轨迹缓解先进AI系统控制丧失
Mitigating loss of control in advanced AI systems through instrumental goal trajectories
AI总结:
本文提出通过组织路径中的采购、治理和财务工具性目标轨迹来缓解先进AI系统控制丧失问题。
摘要:
人工智能实验室和大学的研究人员担忧高度 capable 的人工智能系统可能通过追求工具性目标侵蚀人类控制。现有的缓解措施仍然主要是技术性和系统性的:在高级系统中跟踪能力,通过诸如人类反馈强化学习等方法塑造行为,以及设计系统使其可纠正和可中断。在此,我们开发了工具性目标轨迹,以扩展这些选项超出模型本身。获得能力通常取决于对额外技术资源的访问,如计算、存储、数据和相邻服务,这些又需要对货币资源的访问。在组织中,这些资源可以通过三种组织路径获得。我们称这些路径为采购、治理和财务工具性目标轨迹(IGTs)。每个IGT会产生一系列组织 artifacts,可以在系统能力或行为超过可接受阈值时被监控并用作干预点。通过这种方式,IGTs为定义能力水平和扩大可纠正性和可中断性实施提供了具体途径,将关注点从模型属性本身转向使它们成为可能的组织系统。
英文摘要:
Researchers at artificial intelligence labs and universities are concerned that highly capable artificial intelligence (AI) systems may erode human control by pursuing instrumental goals. Existing mitigations remain largely technical and system-centric: tracking capability in advanced systems, shaping behaviour through methods such as reinforcement learning from human feedback, and designing systems to be corrigible and interruptible. Here we develop instrumental goal trajectories to expand these options beyond the model. Gaining capability typically depends on access to additional technical resources, such as compute, storage, data and adjacent services, which in turn requires access to monetary resources. In organisations, these resources can be obtained through three organisational pathways. We label these pathways the procurement, governance and finance instrumental goal trajectories (IGTs). Each IGT produces a trail of organisational artefacts that can be monitored and used as intervention points when a systems capabilities or behaviour exceed acceptable thresholds. In this way, IGTs offer concrete avenues for defining capability levels and for broadening how corrigibility and interruptibility are implemented, shifting attention from model properties alone to the organisational systems that enable them.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。