静默更新：测量并弥合部署后的披露差距

中文摘要 AI 辅助

部署后的基础模型并非静态系统，提供商可通过微调、分类器更新、系统提示修订、检索调整及路由变更等方式修改系统行为。此类更新可静默进行，即不公开披露、不更新版本号、不重新评估。这种静默更新挑战了当前AI治理框架的核心假设：外部可验证的保管链将评估结果或系统卡片中提及的模型与提供给用户的模型关联起来。本文研究了第一方API提供商和推理主机的部署后披露实践，以确定实际中保管链的存在程度。研究发现，提供商通常会发布大量安全文档，包括定量评估和特定版本报告，但在我们的样本中，没有任何提供商发布信息允许外部方验证所提供的制品是否与文档中提及的模型一致。我们推出了静默更新评分卡（Silent Updates Scorecard），这是一种用于测量各提供商和主机部署后披露实践的公开工具，并给出了9家第一方API提供商和7家第三方推理主机样本的初步结果。我们还提出了三部分行为触发系统，用于确定对系统的部署后修改何时会触发披露或重新评估义务。

英文摘要

Deployed foundation models are often not static systems, with providers able to modify system behavior through fine-tuning, classifier updates, system prompt revisions, retrieval changes, and routing changes. These updates can be made silently -- that is, without public disclosure, a version increment, or re-evaluation. Such silent updates challenge a core assumption behind current AI governance frameworks that an externally verifiable chain of custody links the model referred to in evaluation results or a system card to the model served to users. In this paper, we examine post-deployment disclosure practices across first-party API providers and inference hosts to establish the extent to which a chain of custody exists in practice. We find that providers commonly publish substantial safety documentation, including quantitative evaluations and version-specific reports, but no provider in our sample published information allowing an external party to verify that the artifact being served is the same one referred to in this documentation. We introduce the Silent Updates Scorecard, a public instrument for measuring post-deployment disclosure practices across providers and hosts, and preliminary results for a sample of nine first-party API providers and seven third-party inference hosts. We also propose a Three-Part Behavioral Trigger System for determining when post-deployment modifications to a system motivate disclosure or re-evaluation obligations.

补充信息

备注

Accepted to AIES-26