从档案伪影中解构算法偏见：大都会博物馆档案中视觉语言模型评估的受控审计
Disentangling Algorithmic Bias from Archival Artifacts: A Controlled Audit of Vision-Language Model Valuation in Metropolitan Museum Archives
查看机构详情
- Boston University（波士顿大学）
- University of Eastern Finland（东芬兰大学）
- Symbiosis Institute of Technology, Pune, Symbiosis International (Deemed University)（浦那共生技术学院，共生国际（被视为大学））
机构由 AI 辅助整理，请以论文原文为准。
浏览论文内容
中文总结 AI 辅助
本研究通过控制混杂因素审计CLIP模型在大都会艺术博物馆档案中的性别偏见，发现宏观分数等价但存在测量局限，强调多变量控制和档案审计的必要性。
中文摘要 AI 辅助
审计视觉语言模型（VLM）的社会偏见，需要区分直接的算法评估差异与嵌入在档案元数据中的混杂因素。在本研究中，我们使用大都会艺术博物馆开放获取藏品的历史艺术品元数据（总对象数N = 1,500；其中署名作品N = 743：男性n = 534，女性n = 209；匿名作品n = 618）对对比语言-图像预训练（CLIP）模型进行了审计。我们建立了一个定量审计框架，评估三个语义提示对（杰作、质量和影响力）下的零样本CLIP logit差分分数。未经调整的评估显示，在OpenAI CLIP（mu_F = -0.0067 vs mu_M = -0.0035，p = 0.1829）或OpenCLIP（mu_F = 0.0171 vs mu_M = 0.0237，p = 0.1224）下，得分高度收敛，且无统计学显著的主要性别效应。双单侧检验（TOST）确认在Cohen's d >= 0.25界限内具有统计等价性（pTOST < 0.005）。控制艺术品媒介、创作时代和宽高比的多变量OLS回归（R^2 < 0.02）确认艺术家性别无统计学显著的条件效应（p > 0.20）。高残差嵌入方差（R^2 < 2%）表明，全局零样本评估指标在嵌入噪声基底附近运行，显示广泛的零样本提示logit差分是一种粗糙的测量工具，而非证明模型的绝对公平性。我们强调两个关键注意事项：（i）宏观层面的分数等价性反映了指标对细粒度视觉语义特征的不敏感性，并不排除局部微观层面的视觉偏见；（ii）排除41.2%的未署名藏品反映了制度性生存偏见。这些结果证明了在评估文化遗产藏品中AI公平性时，多变量混杂控制、等价性检验和档案来源审计的必要性。
英文摘要
Auditing vision-language models (VLMs) for societal bias requires distinguishing direct algorithmic valuation disparities from confounders embedded within archival metadata. In this study, we audit Contrastive Language-Image Pretraining (CLIP) models using historical artwork metadata from the Metropolitan Museum of Art Open Access collection (N = 1,500 total objects; N = 743 attributed works: Male n = 534, Female n = 209; n = 618 anonymous). We establish a quantitative audit framework evaluating zero-shot CLIP logit differential scores across three semantic prompt pairs (masterpiece, quality, and influence). Unadjusted evaluations demonstrate high score convergence without a statistically significant main gender effect under OpenAI CLIP (mu_F = -0.0067 vs mu_M = -0.0035, p = 0.1829) or OpenCLIP (mu_F = 0.0171 vs mu_M = 0.0237, p = 0.1224). Two One-Sided Tests (TOST) confirm statistical equivalence across Cohen's d >= 0.25 bounds (pTOST < 0.005). Multivariate OLS regression controlling for artwork medium, creation era, and aspect ratio (R^2 < 0.02) confirms that artist gender has no statistically significant conditional effect (p > 0.20). High residual embedding variance (R^2 < 2%) indicates that global zero-shot valuation metrics operate near an embedding noise floor, showing that broad zero-shot prompt logit differentials are a coarse measurement instrument rather than proving absolute model fairness. We highlight two key caveats: (i) macro-level score equivalence reflects metric insensitivity to fine-grained visual-semantic features and does not preclude localized micro-level visual biases, and (ii) excluding 41.2% unattributed holdings reflects institutional survival bias. These results demonstrate the necessity of multivariate confound control, equivalence testing, and archival provenance auditing when assessing AI fairness in cultural heritage collections.
