如果自动化AI研发引发智能爆炸会怎样？

中文摘要 AI 辅助

与一年前相比，如今AI系统在构建它们的公司内部编写了大部分代码。随着越来越多的AI研究与开发（R&D）流程被自动化，AI的进步是否会在“智能爆炸”中急剧加速，将多年的进展压缩到几个月甚至更短？初步证据表明这有可能。在这项工作中，我们评估了这一证据，分析了智能爆炸的潜在影响，并提出了政策应对措施。AI系统有望在几年内自动化大部分AI研发工作，甚至可能全部自动化。如果这引发智能爆炸，它可能大幅提前AI带来的益处，但也带来极端风险：能力增长可能加速到远超社会所能跟上的程度，人类可能失去对超人类AI系统的控制，国家、公司及政府分支内部和之间的权力制衡可能被严重削弱。尽管这些可能性仍存在诸多不确定性，但高风险值得进一步严肃关注。政策制定者应紧急获取更多关于AI研发自动化的可见性，开发引导和约束智能爆炸的方法，并准备社会适应智能爆炸的影响。

英文摘要

In contrast to even a year ago, AI systems now write most of the code inside the companies that build them. As more of the AI research and development (R&D) pipeline is automated, could AI progress radically accelerate in an "intelligence explosion," where years of advances are compressed into months or less? Preliminary evidence suggests that it could. In this work, we assess this evidence, analyze an intelligence explosion's potential impacts, and propose policy responses. AI systems are on track to automate most AI R\&D work within a few years, and possibly all of it. If this triggers an intelligence explosion, it could dramatically bring forward AI's benefits, but also pose extreme risks: capabilities growth could accelerate far beyond what society can keep up with, humanity could lose control over superhuman AI systems, and checks on power within and between states, companies, and branches of government could be severely eroded. Although there remains much uncertainty about these possibilities, the high stakes warrant serious further attention. Policymakers should urgently obtain more visibility into the automation of AI R&D, develop ways to steer and constrain an intelligence explosion, and prepare society to adapt to an intelligence explosion's impacts.

发表机构

• GovAI
• CASP, University of Cambridge（剑桥大学CASP）
• Institute for Law & AI（法律与人工智能研究所）
• University of Massachusetts Amherst（马萨诸塞大学阿默斯特分校）
• OpenAI
• University of Toronto（多伦多大学）
• Vector Institute（矢量研究所）
• Microsoft（微软）
• Mila (Quebec AI Institute)（Mila（魁北克人工智能研究所））
• LawZero
• University of California, Berkeley（加州大学伯克利分校）
• Anthropic
• University of Oxford（牛津大学）
• University of Virginia（弗吉尼亚大学）
• Foundation for American Innovation（美国创新基金会）
• University College London（伦敦大学学院）
• University of British Columbia（不列颠哥伦比亚大学）
• Guidelight
• Forethought
• AI Policy Institute（人工智能政策研究所）

机构由 AI 辅助整理，请以论文原文为准。