AI 中文总结
本文指出开放权重AI模型风险管理存在16项涉及多环节的技术挑战，强调相关研究需兼顾开放性，以实现其益处并减轻危害。
AI 中文摘要
拥有公开权重的前沿AI模型正稳步变得更强大且被广泛采用，但与专有模型相比，开放权重模型带来了不同的机遇与挑战。例如，它们支持更开放的研究与测试；然而，其风险管理颇具挑战，因为可被任意修改、在无监督情况下使用且会不可逆传播。目前，针对开放权重模型安全的工具类研究有限。解决这些缺口对实现其益处并减轻危害至关重要。本文提出了开放权重模型安全领域的16个开放技术挑战，涉及训练数据、训练算法、评估、部署及生态系统监测。最后，本文探讨了该领域的新兴状态，强调研究、方法与评估的开放性——而非仅权重——将是构建严谨的开放权重模型风险管理科学的关键。
英文摘要
Frontier AI models with openly available weights are steadily becoming more powerful and widely adopted. However, compared to proprietary models, open-weight models pose different opportunities and challenges for effective risk management. For example, they allow for more open research and testing. However, managing their risks is also challenging because they can be modified arbitrarily, used without oversight, and spread irreversibly. Currently, there is limited research on safety tooling specific to open-weight models. Addressing these gaps will be key to both realizing their benefits and mitigating their harms. In this paper, we present 16 open technical challenges for open-weight model safety involving training data, training algorithms, evaluations, deployment, and ecosystem monitoring. We conclude by discussing the nascent state of the field, emphasizing that openness about research, methods, and evaluations -- not just weights -- will be key to building a rigorous science of open-weight model risk management.
CommentsPublished in Transactions on Machine Learning Research (03/2026) Reviewed on OpenReview: https: // openreview. net/ forum? id= 8QyGLnFkzc
