本立场文件主张,AI 代理应根据其自主操作的程度进行监管。具有长期规划和战略能力的 AI 代理可能带来人类灭绝和不可逆转的全球性灾难的重大风险。虽然现有监管往往侧重于计算规模作为潜在危害的代理指标,但我们认为,对于能力主要源于推理时计算的代理,此类措施不足以评估其风险。为支持我们的立场,我们讨论了科学家关于存在性风险的相关法规和建议,以及使用动作序列(反映代理自主程度)作为比依赖观察环境状态的现有指标更合适的潜在影响衡量标准的优势。
英文摘要:
This position paper argues that AI agents should be regulated by the extent to which they operate autonomously. AI agents with long-term planning and strategic capabilities can pose significant risks of human extinction and irreversible global catastrophes. While existing regulations often focus on computational scale as a proxy for potential harm, we argue that such measures are insufficient for assessing the risks posed by agents whose capabilities arise primarily from inference-time computation. To support our position, we discuss relevant regulations and recommendations from scientists regarding existential risks, as well as the advantages of using action sequences -- which reflect the degree of an agent's autonomy -- as a more suitable measure of potential impact than existing metrics that rely on observing environmental states.
Right in Time: Reactive Reasoning in Regulated Traffic Spaces
Artificial Intelligence and Machine Learning Lab, Department of Computer Science, TU Darmstadt(人工智能与机器学习实验室,计算机科学系,图林根大学);Konrad Zuse School of Excellence in Learning and Intelligent Systems(Konrad Zuse 学校,卓越学习与智能系统);Honda Research Institute Europe GmbH(本田欧洲研究院);Hessian Center for Artificial Intelligence (hessian.AI)(黑森人工智能中心);Centre for Cognitive Science(认知科学中心);German Center for Artificial Intelligence (DFKI)(德国人工智能中心);Uncertainty in Artificial Intelligence Group, Department of Mathematics and Computer Science, TU Eindhoven(人工智能不确定性小组,数学与计算机科学系,埃因霍温理工大学)
Continuous Time Continuous Space Homeostatic Reinforcement Learning (CTCS-HRRL) : Towards Biological Self-Autonomous Agent
Ecole Normale Superieure(巴黎高等师范学院);BITS Pilani K. K. Birla Goa Campus(比拉理工学院K. K. Birla果阿校区);Ecole Normale Supérieure Paris-Saclay(巴黎-萨克雷高等师范学院);Université Paris-Saclay(巴黎-萨克雷大学)