AI带来的灭绝风险:科学无法观测?
Extinction Risks from AI: Invisible to Science?
AI总结:
本研究提出灭绝级古德哈特定律假设,明确了评估该假设的模型需满足的必要条件,指出模型复杂度或导致AI灭绝风险的底层机制难以被现有科学方法探究。
摘要:
为给围绕AI存在性风险的讨论提供参考,我们提出“灭绝级古德哈特定律”,其表述为“几乎任何目标设定,若被极端追求,都将导致人类灭绝”,并旨在探究哪些形式化模型适合研究这一假设。注意我们对灭绝级古德哈特定律是否成立持不可知态度。作为核心贡献,我们确定了一组必要条件,一个模型若要能有效评估支持灭绝级古德哈特定律的具体论证,就必须满足这些条件。由于每个条件似乎都会显著增加所得模型的复杂度,对该假设进行形式化评估可能极其困难。这引出了一种可能性:无论AI带来的灭绝风险是否真实存在,其背后的动态机制可能都无法被当前科学方法观测到。
英文摘要:
In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart's Law as "Virtually any goal specification, pursued to the extreme, will result in the extinction of humanity", and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart's Law holds or not. As our key contribution, we identify a set of conditions that are necessary for a model that aims to be informative for evaluating specific arguments for Extinction-level Goodhart's Law. Since each of the conditions seems to significantly contribute to the complexity of the resulting model, formally evaluating the hypothesis might be exceedingly difficult. This raises the possibility that whether the risk of extinction from artificial intelligence is real or not, the underlying dynamics might be invisible to current scientific methods.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。