在异构AI生态系统中量化模型独特性
Quantifying Model Uniqueness in Heterogeneous AI Ecosystems
AI总结:
本文提出基于ISQED的统计框架,用于量化异构AI生态系统中模型的独特性,通过PIER指标评估模型行为的不可约简性,并建立理论基础与实验验证。
摘要:
随着AI系统从孤立的预测器发展为复杂、异构的基础模型和专用适配器生态系统,区分真正的行为新颖性与功能冗余成为关键的治理挑战。本文介绍了一种基于In-Silico Quasi-Experimental Design(ISQED)的统计框架,用于审计模型独特性。通过在模型间强制匹配干预,我们隔离了内在模型身份,并将独特性量化为Peer-Inexpressible Residual(PIER),即目标行为中严格不可约简为任何随机凸组合的同伴行为的组成部分,PIER趋于零时表示基于路由的替代成为可能。我们通过三个关键贡献建立了生态系统审计的理论基础。首先,我们证明了观测日志的根本限制:在无干预控制的情况下,独特性在数学上无法辨识。其次,我们推导了主动审计的缩放定律,证明我们的自适应查询协议实现了最小最大最优的样本效率(dσ²γ⁻²log(Nd/δ))。第三,我们证明了合作博弈论方法,如Shapley值,本质上无法检测冗余。我们通过DISCO(Design-Integrated Synthetic Control)估计器实现该框架,并在多样化的生态系统中部署,包括计算机视觉模型(ResNet/ConvNeXt/ViT)、大型语言模型(BERT/RoBERTa)和城市级交通预测器。这些结果使可信AI超越了解释单个模型:它们确立了以干预为基础的审计和治理异构模型生态系统的原理性科学。
英文摘要:
As AI systems evolve from isolated predictors into complex, heterogeneous ecosystems of foundation models and specialized adapters, distinguishing genuine behavioral novelty from functional redundancy becomes a critical governance challenge. Here, we introduce a statistical framework for auditing model uniqueness based on In-Silico Quasi-Experimental Design (ISQED). By enforcing matched interventions across models, we isolate intrinsic model identity and quantify uniqueness as the Peer-Inexpressible Residual (PIER), i.e. the component of a target's behavior strictly irreducible to any stochastic convex combination of its peers, with vanishing PIER characterizing when such a routing-based substitution becomes possible. We establish the theoretical foundations of ecosystem auditing through three key contributions. First, we prove a fundamental limitation of observational logs: uniqueness is mathematically non-identifiable without intervention control. Second, we derive a scaling law for active auditing, showing that our adaptive query protocol achieves minimax-optimal sample efficiency ($dσ^2γ^{-2}\log(Nd/δ)$). Third, we demonstrate that cooperative game-theoretic methods, such as Shapley values, fundamentally fail to detect redundancy. We implement this framework via the DISCO (Design-Integrated Synthetic Control) estimator and deploy it across diverse ecosystems, including computer vision models (ResNet/ConvNeXt/ViT), large language models (BERT/RoBERTa), and city-scale traffic forecasters. These results move trustworthy AI beyond explaining single models: they establish a principled, intervention-based science of auditing and governing heterogeneous model ecosystems.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。