谁在评估人工智能的社会影响？第一方和第三方评估的覆盖范围与差距分析
Who Evaluates AI's Social Impacts? Mapping Coverage and Gaps in First and Third Party Evaluations
AI总结：
通过分析186份第一方发布报告和248份第三方评估来源，结合开发者访谈，揭示了第一方报告稀疏且流于表面，而第三方评估更广泛深入，但数据溯源、内容审核劳动等关键领域存在披露缺口，呼吁政策强制开发者透明化并加强独立评估生态。
摘要：
基础模型日益成为高风险人工智能系统的核心，治理框架现在依赖评估来评估其风险和能力。尽管通用能力评估已广泛开展，但涵盖偏见、公平性、隐私、环境成本和劳动的社会影响评估仍不均衡。为了描述这一格局，我们进行了首次社会影响评估报告的综合分析，检查了186份第一方发布报告和248份第三方评估来源，并辅以开发者访谈。我们发现明显的分工：第一方报告稀疏、通常流于表面，且在环境影响和偏见等领域呈下降趋势，而第三方评估者提供了更广泛、更严格的偏见、有害内容和性能差异覆盖。然而，只有开发者才能权威地报告数据来源、内容审核劳动、成本和基础设施，但访谈揭示这些披露除非与产品采用或合规挂钩，否则被降级优先。当前实践在评估社会影响方面留下了重大空白，强调了需要制定政策强制开发者透明化、加强独立评估生态系统，并创建聚合第三方评估的共享基础设施。
英文摘要：
Foundation models are increasingly central to high-stakes AI systems, and governance frameworks now depend on evaluations to assess their risks and capabilities. Although general capability evaluations are widespread, social impact assessments covering bias, fairness, privacy, environmental costs, and labor remain uneven. To characterize this landscape, we conduct the first comprehensive analysis of social impact evaluation reporting, examining 186 first-party release reports and 248 third-party evaluation sources, supplemented by developer interviews. We find a stark division of labor: first-party reporting is sparse, often superficial, and declining in areas like environmental impact and bias, while third-party evaluators provide broader, more rigorous coverage of bias, harmful content, and performance disparities. However, only developers can authoritatively report on data provenance, content moderation labor, costs, and infrastructure, yet interviews reveal these disclosures are deprioritized unless tied to product adoption or compliance. Current practices leave major gaps in assessing societal impacts, underscoring the need for policies that mandate developer transparency, strengthen independent evaluation ecosystems, and create shared infrastructure for aggregating third-party evaluations.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
