arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2310.18244cs.CYcs.AI
收藏

关于人工智能通过失准的权力寻求导致生存风险的证据综述

A Review of the Evidence for Existential Risk from AI via Misaligned Power-Seeking

AI Impacts(AI影响研究所)

Rose Hadshar

AI总结:

本文综述了AI通过目标失准和权力寻求导致生存风险的证据,发现规范博弈有强实证支持而失准权力寻求尚无公开实例,结论是虽证据尚不确定但无法排除该风险,值得严重关切。

摘要:

人工智能(AI)的快速发展引发了专家、政策制定者和世界领导人日益增长的担忧,即越来越先进的AI系统可能带来生存风险。本文综述了AI通过失准(AI系统发展出与人类价值观不一致的目标)和权力寻求(失准的AI主动寻求权力)导致生存风险的证据。本综述考察了与规范博弈(specification gaming)、目标泛化错误(goal misgeneralization)和权力寻求相关的实证发现、概念论证和专家意见。现有证据状况令人担忧,但关于极端形式的失准权力寻求是否存在,尚无定论。规范博弈的有力实证证据与权力寻求的有力概念证据相结合,使得难以排除失准权力寻求带来生存风险的可能性。另一方面,迄今为止尚无AI系统中失准权力寻求的公开实证案例,因此关于未来系统将构成生存风险的论证在某种程度上仍属推测。鉴于现有证据状况,很难对失准权力寻求构成巨大生存风险或完全不构成生存风险抱有极大信心。我们无法自信地排除AI通过失准权力寻求导致生存风险这一事实,值得引起严重关切。

英文摘要:

Rapid advancements in artificial intelligence (AI) have sparked growing concerns among experts, policymakers, and world leaders regarding the potential for increasingly advanced AI systems to pose existential risks. This paper reviews the evidence for existential risks from AI via misalignment, where AI systems develop goals misaligned with human values, and power-seeking, where misaligned AIs actively seek power. The review examines empirical findings, conceptual arguments and expert opinion relating to specification gaming, goal misgeneralization, and power-seeking. The current state of the evidence is found to be concerning but inconclusive regarding the existence of extreme forms of misaligned power-seeking. Strong empirical evidence of specification gaming combined with strong conceptual evidence for power-seeking make it difficult to dismiss the possibility of existential risk from misaligned power-seeking. On the other hand, to date there are no public empirical examples of misaligned power-seeking in AI systems, and so arguments that future systems will pose an existential risk remain somewhat speculative. Given the current state of the evidence, it is hard to be extremely confident either that misaligned power-seeking poses a large existential risk, or that it poses no existential risk. The fact that we cannot confidently rule out existential risk from AI via misaligned power-seeking is cause for serious concern.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑