Supervision policies can shape long-term risk management in general-purpose AI models
Department of Statistics, Universidad Carlos III de Madrid, Spain(马德里卡洛斯三世大学统计系);Center for Automation and Robotics, Spanish National Research Council, Madrid, Spain(西班牙国家研究委员会自动化与机器人中心);European Commission, Joint Research Centre, Seville, Spain(欧盟委员会联合研究中心);Computer Engineering Department, University of Alcalá, Alcalá de Henares, Spain(阿尔卡拉大学计算机工程系)
Manuel Cebrian, Emilia Gomez, David Fernandez Llorca
本文构建风险报告仿真框架并结合真实 ChatGPT 数据,比较四种 AI 监管政策,揭示优先级策略虽能缓解高影响风险,却可能造成反馈循环和风险认知偏差。
摘要:
包括大语言模型(LLM)在内的通用人工智能(GPAI)模型的快速扩散与部署,给 AI 监管机构带来了前所未有的挑战。我们假设,这些机构将需要应对一个新兴的风险与事件报告生态系统,而其规模很可能超出监管能力。为研究这一点,我们开发了一个仿真框架,其参数取自风险、事件或危害报告生态系统多元图景中的特征,包括社区驱动平台、众包举措和专家评估。我们评估了四种监管政策:非优先级(先到先服务)、随机选择、基于优先级(首先处理最高优先级风险)以及多样性优先(在高优先级风险与覆盖各类风险的全面性之间取得平衡)。结果表明,尽管基于优先级和多样性优先的政策在缓解高影响风险(尤其是专家识别出的风险)方面更有效,但它们可能在无意中忽视更广泛社区报告的系统性问题。这种忽视会形成反馈循环,放大某些类型的报告,同时抑制其他类型,从而导致对整体风险图景的认知偏差。我们使用多个真实世界数据集验证了仿真结果,其中一个数据集包含超过一百万次 ChatGPT 交互,并有超过 15 万段对话被识别为存在风险。该验证凸显了 AI 风险监管内在的复杂权衡,并表明风险管理政策的选择如何能够塑造社会中使用的各类 GPAI 模型未来的 AI 风险图景。
英文摘要:
The rapid proliferation and deployment of General-Purpose AI (GPAI) models, including large language models (LLMs), present unprecedented challenges for AI supervisory entities. We hypothesize that these entities will need to navigate an emergent ecosystem of risk and incident reporting, likely to exceed their supervision capacity. To investigate this, we develop a simulation framework parameterized by features extracted from the diverse landscape of risk, incident, or hazard reporting ecosystems, including community-driven platforms, crowdsourcing initiatives, and expert assessments. We evaluate four supervision policies: non-prioritized (first-come, first-served), random selection, priority-based (addressing the highest-priority risks first), and diversity-prioritized (balancing high-priority risks with comprehensive coverage across risk types). Our results indicate that while priority-based and diversity-prioritized policies are more effective at mitigating high-impact risks, particularly those identified by experts, they may inadvertently neglect systemic issues reported by the broader community. This oversight can create feedback loops that amplify certain types of reporting while discouraging others, leading to a skewed perception of the overall risk landscape. We validate our simulation results with several real-world datasets, including one with over a million ChatGPT interactions, of which more than 150,000 conversations were identified as risky. This validation underscores the complex trade-offs inherent in AI risk supervision and highlights how the choice of risk management policies can shape the future landscape of AI risks across diverse GPAI models used in society.
Fudan University(复旦大学);Shanghai Key Laboratory of Data Science(上海市数据科学重点实验室);College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学与人工智能学院)
Optimizing Crop Management with Reinforcement Learning and Imitation Learning
University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of Tasmania(塔斯马尼亚大学);Stockholm University(斯德哥尔摩大学);KTH Royal Institute of Technology(瑞典皇家理工学院)
A Semantic Framework for Enabling Radio Spectrum Policy Management and Evaluation
Rensselaer Polytechnic Institute(伦斯勒理工学院);Memory Based Research LLC(Memory Based Research有限责任公司);The Rensselaer Institute for Data Exploration and Applications(伦斯勒数据探索与应用研究所);Capraro Technologies Inc.(Capraro科技公司);CACI International Inc.(CACI国际公司);Air Force Research Laboratory(空军研究实验室)
Higher-Order Action Supervision Makes A Strong Policy Class
Institute for AI Industry Research (AIR), Tsinghua University(清华大学人工智能产业研究院(AIR));Zenfex AI Lab(Zenfex人工智能实验室);University of Electronic Science and Technology of China(电子科技大学);Horizon Robotics(地平线机器人)