arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2206.05862cs.CYcs.AIcs.LG
收藏

X-AI研究的风险分析

X-Risk Analysis for AI Research

UC Berkeley(加州大学伯克利分校);UIUC(伊利诺伊大学厄巴纳-香槟分校)

Dan Hendrycks, Mantas Mazeika

摘要:

人工智能(AI)有可能极大地改善社会,但与任何强大的技术一样,它也带来了更高的风险和责任。目前的人工智能研究缺乏对如何管理人工智能系统的长尾风险(包括投机性长期风险)的系统性讨论。考虑到人工智能的潜在好处,有人担心,构建越来越智能和强大的人工智能系统最终可能会导致比我们更强大的系统;有人说这就像玩火,并推测这可能会产生存在风险(x风险)。为了增加精确度并为这些讨论奠定基础,我们提供了如何分析AI x风险的指南,该指南由三个部分组成:首先,我们回顾了当今如何使系统变得更安全,借鉴了危险分析和系统安全中经过时间考验的概念,这些概念旨在将大型流程引向更安全的方向。接下来,我们将讨论对未来系统安全性产生长期影响的策略。最后,我们讨论了通过改善安全性和一般功能之间的平衡来使AI系统更安全的一个关键概念。我们希望本文档以及所提出的概念和工具可以作为理解如何分析AI x风险的有用指南。

英文摘要:

Artificial intelligence (AI) has the potential to greatly improve society, but as with any powerful technology, it comes with heightened risks and responsibilities. Current AI research lacks a systematic discussion of how to manage long-tail risks from AI systems, including speculative long-term risks. Keeping in mind the potential benefits of AI, there is some concern that building ever more intelligent and powerful AI systems could eventually result in systems that are more powerful than us; some say this is like playing with fire and speculate that this could create existential risks (x-risks). To add precision and ground these discussions, we provide a guide for how to analyze AI x-risk, which consists of three parts: First, we review how systems can be made safer today, drawing on time-tested concepts from hazard analysis and systems safety that have been designed to steer large processes in safer directions. Next, we discuss strategies for having long-term impacts on the safety of future systems. Finally, we discuss a crucial concept in making AI systems safer by improving the balance between safety and general capabilities. We hope this document and the presented concepts and tools serve as a useful guide for understanding how to analyze AI x-risk.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑