推理如何影响视觉语言模型中的交叉偏见
How Reasoning Influences Intersectional Biases in Vision Language Models
AI总结：
本文在 FairFace 数据集上对五个开源视觉语言模型进行职业预测实验，系统分析推理过程，发现偏见性推理模式是交叉差异的系统性根源，强调部署前需使 VLM 推理与人类价值观对齐。
摘要：
视觉语言模型（VLMs）越来越多地被部署到下游任务中，但其训练数据往往编码了会在输出中显现的社会偏见。与人类通过情境和社会线索来解读图像不同，VLMs 通过统计关联来处理图像，这常常导致其推理偏离人类推理。通过分析 VLM 的推理方式，我们可以理解固有偏见如何被延续，并可能对下游性能产生不利影响。为考察这一差距，我们在 FairFace 数据集上，针对职业预测任务，系统分析了五个开源 VLM 中的社会偏见。在 32 种职业和三种不同提示风格下，我们同时获取了模型的预测和推理。我们的发现表明，有偏的推理模式系统性地构成了交叉差异的基础，突显了在将 VLM 推理部署到下游之前，使其与人类价值观保持一致的必要性。
英文摘要：
Vision Language Models (VLMs) are increasingly deployed across downstream tasks, yet their training data often encode social biases that surface in outputs. Unlike humans, who interpret images through contextual and social cues, VLMs process them through statistical associations, often leading to reasoning that diverges from human reasoning. By analyzing how a VLM reasons, we can understand how inherent biases are perpetuated and can adversely affect downstream performance. To examine this gap, we systematically analyze social biases in five open-source VLMs for an occupation prediction task, on the FairFace dataset. Across 32 occupations and three different prompting styles, we elicit both predictions and reasoning. Our findings reveal that the biased reasoning patterns systematically underlie intersectional disparities, highlighting the need to align VLM reasoning with human values prior to its downstream deployment.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
