arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2412.08653cs.CYcs.AI
收藏

用于防范灾难性风险的AI评估能做什么与不能做什么

What AI evaluations for preventing catastrophic risks can and cannot do

Machine Intelligence Research Institute(机器智能研究所)

Peter Barnett, Lisa Thiergart

AI总结:

本文探讨了用于防范灾难性风险的AI评估的作用与局限,指出其可确定AI能力下界、评估部分滥用风险,但存在无法确定能力上界等根本缺陷,提出不应将其作为AI安全的主要保障,并给出前沿AI安全的渐进改进建议。

摘要:

AI评估是AI治理工具体系中的重要组成部分,是当前用于构建防范灾难性风险安全论证的方法的基础。本文研究了这类评估能够以及不能够向我们揭示的内容。评估可以确定AI能力的下界,并且在评估者投入足够精力的前提下,评估特定的滥用风险。\n遗憾的是,评估面临着在当前范式下无法克服的根本性局限,包括无法确定能力的上界、无法可靠预测未来模型的能力,也无法稳健评估自主AI系统带来的风险。这意味着尽管评估是有价值的工具,我们不应将其作为确保AI系统安全的主要手段。本文最后提出了前沿AI安全的渐进式改进建议,同时承认这些根本性局限仍未得到解决。

英文摘要:

AI evaluations are an important component of the AI governance toolkit, underlying current approaches to safety cases for preventing catastrophic risks. Our paper examines what these evaluations can and cannot tell us. Evaluations can establish lower bounds on AI capabilities and assess certain misuse risks given sufficient effort from evaluators. Unfortunately, evaluations face fundamental limitations that cannot be overcome within the current paradigm. These include an inability to establish upper bounds on capabilities, reliably forecast future model capabilities, or robustly assess risks from autonomous AI systems. This means that while evaluations are valuable tools, we should not rely on them as our main way of ensuring AI systems are safe. We conclude with recommendations for incremental improvements to frontier AI safety, while acknowledging these fundamental limitations remain unsolved.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑