超级智能无法遏制:来自可计算性理论的教训
Superintelligence cannot be contained: Lessons from Computability Theory
摘要:
超级智能是一种假想的智能体,它拥有的智能远远超过最聪明、最有天赋的人类大脑。鉴于机器智能的最新进展,一些科学家、哲学家和技术专家重新开始讨论这种实体所带来的潜在灾难性风险。在这篇文章中,我们追溯了对超级智能的新恐惧的起源和发展,以及遏制它的一些主要建议。我们认为,这种遏制是,在原则上,不可能的,由于计算本身固有的基本限制。假设一个超级智能将包含一个程序,该程序包括所有可以由通用图灵机执行的程序,这些程序的输入可能与世界的状态一样复杂,严格的遏制需要模拟这样一个程序,这在理论上(和实际上)是不可行的。
英文摘要:
Superintelligence is a hypothetical agent that possesses intelligence far surpassing that of the brightest and most gifted human minds. In light of recent advances in machine intelligence, a number of scientists, philosophers and technologists have revived the discussion about the potential catastrophic risks entailed by such an entity. In this article, we trace the origins and development of the neo-fear of superintelligence, and some of the major proposals for its containment. We argue that such containment is, in principle, impossible, due to fundamental limits inherent to computing itself. Assuming that a superintelligence will contain a program that includes all the programs that can be executed by a universal Turing machine on input potentially as complex as the state of the world, strict containment requires simulations of such a program, something theoretically (and practically) infeasible.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。