arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2503.04750cs.CYcs.AI
收藏

AI 代理应根据其自主操作程度进行监管

AI Agents Should be Regulated Based on the Extent of Their Autonomous Operations

IBM Research – Tokyo(IBM东京研究院)

Takayuki Osogami

AI总结:

本文主张根据AI代理的自主操作程度进行监管,提出动作序列作为衡量潜在影响的更合适指标,以应对长期规划和战略能力带来的存在性风险。

摘要:

本立场文件主张,AI 代理应根据其自主操作的程度进行监管。具有长期规划和战略能力的 AI 代理可能带来人类灭绝和不可逆转的全球性灾难的重大风险。虽然现有监管往往侧重于计算规模作为潜在危害的代理指标,但我们认为,对于能力主要源于推理时计算的代理,此类措施不足以评估其风险。为支持我们的立场,我们讨论了科学家关于存在性风险的相关法规和建议,以及使用动作序列(反映代理自主程度)作为比依赖观察环境状态的现有指标更合适的潜在影响衡量标准的优势。

英文摘要:

This position paper argues that AI agents should be regulated by the extent to which they operate autonomously. AI agents with long-term planning and strategic capabilities can pose significant risks of human extinction and irreversible global catastrophes. While existing regulations often focus on computational scale as a proxy for potential harm, we argue that such measures are insufficient for assessing the risks posed by agents whose capabilities arise primarily from inference-time computation. To support our position, we discuss relevant regulations and recommendations from scientists regarding existential risks, as well as the advantages of using action sequences -- which reflect the degree of an agent's autonomy -- as a more suitable measure of potential impact than existing metrics that rely on observing environmental states.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑