打破语言壁垒还是强化偏见？多语言对比视觉语言模型中的性别与种族差异研究
Breaking Language Barriers or Reinforcing Bias? A Study of Gender and Racial Disparities in Multilingual Contrastive Vision Language Models
AI总结：
本文首次系统审计四种公开多语言CLIP变体的种族与性别偏见，发现多语言性未缓解偏见反而存在刻板印象跨语言迁移等问题，指出需开展语言感知的细粒度偏见评估。
摘要：
多语言视觉语言模型（VLM）有望实现通用图文检索，但其社会偏见尚未得到充分探究。我们首次对M-CLIP、NLLB-CLIP、CAPIVARA-CLIP以及去偏版本SigLIP-2这四种公开多语言CLIP变体开展系统性审计，覆盖10种在资源可用性、形态性别标记上存在差异的语言。我们在零样本设置下使用FairFace的平衡子集与PATA刻板印象套件，量化种族与性别偏见并衡量刻板印象放大程度。与多语言性能缓解偏见的直觉相反，所有模型的性别偏差均强于其仅支持英语的基线版本。CAPIVARA-CLIP恰恰在其面向的低资源语言中表现出最大偏见，而NLLB-CLIP与SigLIP-2的共享编码器会将英语性别刻板印象迁移至无性别标记语言，松耦合编码器则基本可避免这类泄露。尽管SigLIP-2降低了能动性与社群性偏差，但它继承了英语锚点的犯罪关联，甚至在描述稀疏语境（如科萨语）中放大该关联。高性别标记语言会持续放大所有类型的偏见，而当跨语言权重共享引入外来刻板印象时，无性别标记语言仍易受影响。聚合指标会掩盖特定语言的偏见高发点，凸显了未来多语言VLM研究中开展细粒度、语言感知型偏见评估的必要性。
英文摘要：
Multilingual vision-language models (VLMs) promise universal image-text retrieval, yet their social biases remain underexplored. We perform the first systematic audit of four public multilingual CLIP variants: M-CLIP, NLLB-CLIP, CAPIVARA-CLIP, and the debiased SigLIP-2, covering ten languages that differ in resource availability and morphological gender marking. Using balanced subsets of FairFace and the PATA stereotype suite in a zero-shot setting, we quantify race and gender bias and measure stereotype amplification. Contrary to the intuition that multilinguality mitigates bias, every model exhibits stronger gender skew than its English-only baseline. CAPIVARA-CLIP shows its largest biases precisely in the low-resource languages it targets, while the shared encoder of NLLB-CLIP and SigLIP-2 transfers English gender stereotypes into gender-neutral languages; loosely coupled encoders largely avoid this leakage. Although SigLIP-2 reduces agency and communion skews, it inherits -- and in caption-sparse contexts (e.g., Xhosa) amplifies -- the English anchor's crime associations. Highly gendered languages consistently magnify all bias types, yet gender-neutral languages remain vulnerable whenever cross-lingual weight sharing imports foreign stereotypes. Aggregated metrics thus mask language-specific hot spots, underscoring the need for fine-grained, language-aware bias evaluation in future multilingual VLM research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
