2024年基础模型透明度指数
The 2024 Foundation Model Transparency Index
AI总结：
本文介绍FMTI 2024研究，对14家基础模型开发者在100项透明度指标上进行评分，发现平均得分较2023年提升21分至58分，表明透明度可改善且FMTI发挥了促进作用，并建议政策制定者干预未改善领域。
摘要：
基础模型日益重要，却极度不透明。为描述现状，Foundation Model Transparency Index（FMTI，基础模型透明度指数）于2023年10月推出，以衡量领先基础模型开发者的透明度。FMTI 2023在100项透明度指标上评估了10家主要基础模型开发者（如OpenAI、Google）（例如，开发者是否披露其为数据劳动支付的工资？）。当时，开发者公开披露的信息非常有限，平均得分为100分中的37分。为了解现状如何变化，我们在6个月后进行了后续研究：我们用相同的100项指标对14家开发者进行评分。在FMTI 2023中，我们搜索公开可用的信息；而在FMTI 2024中，开发者提交关于100项透明度指标的报告，可能包含此前未公开的信息。我们发现开发者现在的平均得分为100分中的58分，较FMTI 2023提高了21分。这一增长很大程度上源于开发者在FMTI 2024过程中披露信息：平均而言，开发者披露了与16.6项指标相关且此前未公开的信息。我们观察到持续（即跨越2023和2024年）和系统性（即跨越大多数或所有开发者）的不透明区域，如版权状态、数据访问、数据劳动和下游影响。我们发布了整合信息披露的各开发者透明度报告：这些报告基于开发者向我们披露的信息。我们的发现表明，在这个新兴生态系统中透明度可以得到改善，Foundation Model Transparency Index可能促成了这些改进，政策制定者应考虑在透明度未获改善的领域进行干预。
英文摘要：
Foundation models are increasingly consequential yet extremely opaque. To characterize the status quo, the Foundation Model Transparency Index (FMTI) was launched in October 2023 to measure the transparency of leading foundation model developers. FMTI 2023 assessed 10 major foundation model developers (e.g. OpenAI, Google) on 100 transparency indicators (e.g. does the developer disclose the wages it pays for data labor?). At the time, developers publicly disclosed very limited information with the average score being 37 out of 100. To understand how the status quo has changed, we conduct a follow-up study after 6 months: we score 14 developers against the same 100 indicators. While in FMTI 2023 we searched for publicly available information, in FMTI 2024 developers submit reports on the 100 transparency indicators, potentially including information that was not previously public. We find that developers now score 58 out of 100 on average, a 21 point improvement over FMTI 2023. Much of this increase is driven by developers disclosing information during the FMTI 2024 process: on average, developers disclosed information related to 16.6 indicators that was not previously public. We observe regions of sustained (i.e. across 2023 and 2024) and systemic (i.e. across most or all developers) opacity such as on copyright status, data access, data labor, and downstream impact. We publish transparency reports for each developer that consolidate information disclosures: these reports are based on the information disclosed to us via developers. Our findings demonstrate that transparency can be improved in this nascent ecosystem, the Foundation Model Transparency Index likely contributes to these improvements, and policymakers should consider interventions in areas where transparency has not improved.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
