AI转型风险建模（MTAIR）项目--总结报告
Modeling Transformative AI Risks (MTAIR) Project -- Summary Report
摘要：
本报告概述了建模转型人工智能风险（MTAIR）项目的工作，该项目试图在关于先进人工智能灾难性风险的辩论中找出关键假设，不确定性和分歧，以及它们之间的关系。这是建立在本·科蒂埃和罗欣·沙阿早期的图表上的，该图表直观地列出了一些关键的分歧（“症结”），并给出了一些解释。基于广泛的文献综述和专家参与，该报告解释了所涉及问题的模型，以及最初的基于软件的实施，可以将概率估计或其他量化因素纳入其中，以实现探索，规划和/或决策支持。通过将各种辩论和讨论中的信息汇集成一个更连贯的单一陈述，我们希望能够更好地讨论和辩论所涉问题。该模型首先讨论了通过类比推理和关于人工智能的一般先验信念。在此之后，它为高级机器智能提供了一个不同路径和使能技术的模型，以及这些系统的能力如何进步的模型，包括关于自我改进，不连续改进以及分布式，非代理高级智能或较慢改进的可能性的辩论。该模型还特别关注学习优化的问题，以及机器学习系统是否会创建台面优化器。不同的安全研究对前几组问题的影响，然后检查，以了解研究是否以及如何可以在使系统更安全的有用。最后，我们讨论了不同的故障模式和失去控制或接管场景的模型。
英文摘要：
This report outlines work by the Modeling Transformative AI Risk (MTAIR) project, an attempt to map out the key hypotheses, uncertainties, and disagreements in debates about catastrophic risks from advanced AI, and the relationships between them. This builds on an earlier diagram by Ben Cottier and Rohin Shah which laid out some of the crucial disagreements ("cruxes") visually, with some explanation. Based on an extensive literature review and engagement with experts, the report explains a model of the issues involved, and the initial software-based implementation that can incorporate probability estimates or other quantitative factors to enable exploration, planning, and/or decision support. By gathering information from various debates and discussions into a single more coherent presentation, we hope to enable better discussions and debates about the issues involved. The model starts with a discussion of reasoning via analogies and general prior beliefs about artificial intelligence. Following this, it lays out a model of different paths and enabling technologies for high-level machine intelligence, and a model of how advances in the capabilities of these systems might proceed, including debates about self-improvement, discontinuous improvements, and the possibility of distributed, non-agentic high-level intelligence or slower improvements. The model also looks specifically at the question of learned optimization, and whether machine learning systems will create mesa-optimizers. The impact of different safety research on the previous sets of questions is then examined, to understand whether and how research could be useful in enabling safer systems. Finally, we discuss a model of different failure modes and loss of control or takeover scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
