安全洗白：AI安全基准真的能衡量安全进展吗？
Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
AI总结：
本文通过元分析揭示现有AI安全基准与通用能力高度相关，可能导致“安全洗白”，并提出将AI安全定义为可与通用能力进步实证区分的研究目标，以建立更严格的安全评估框架。
摘要：
随着人工智能系统日益强大，人们对“AI安全”研究的兴趣不断增加，以应对新兴和未来的风险。然而，AI安全领域仍然定义不清、测量不一致，导致研究者对如何贡献感到困惑。AI安全基准与上游通用能力（如通用知识和推理）之间关系不明确，进一步加剧了这种模糊性。为了解决这些问题，我们对AI安全基准进行了全面的元分析，在数十个模型上实证分析了它们与通用能力的相关性，并对AI安全领域的现有方向进行了综述。我们的发现揭示，许多安全基准与上游模型能力和训练计算量高度相关，这可能促成“安全洗白”——即能力提升被误表述为安全进步。基于这些发现，我们提出了一个实证基础，用于开发更有意义的安全度量标准，并将机器学习研究背景下的AI安全定义为一组明确界定的研究目标，这些目标在实证上可与通用能力进步区分开来。通过这样做，我们旨在为AI安全研究提供一个更严格的框架，推进安全评估科学，并阐明通往可衡量进展的路径。
英文摘要：
As artificial intelligence systems grow more powerful, there has been increasing interest in "AI safety" research to address emerging and future risks. However, the field of AI safety remains poorly defined and inconsistently measured, leading to confusion about how researchers can contribute. This lack of clarity is compounded by the unclear relationship between AI safety benchmarks and upstream general capabilities (e.g., general knowledge and reasoning). To address these issues, we conduct a comprehensive meta-analysis of AI safety benchmarks, empirically analyzing their correlation with general capabilities across dozens of models and providing a survey of existing directions in AI safety. Our findings reveal that many safety benchmarks highly correlate with both upstream model capabilities and training compute, potentially enabling "safetywashing"--where capability improvements are misrepresented as safety advancements. Based on these findings, we propose an empirical foundation for developing more meaningful safety metrics and define AI safety in a machine learning research context as a set of clearly delineated research goals that are empirically separable from generic capabilities advancements. In doing so, we aim to provide a more rigorous framework for AI safety research, advancing the science of safety evaluations and clarifying the path towards measurable progress.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
