更具典型黑人特征与女性化特征的面孔导致视觉-语言模型中的刻板印象加剧
More Distinctively Black and Feminine Faces Lead to Increased Stereotyping in Vision-Language Models
AI总结：
本研究揭示视觉-语言模型（如GPT-4V）在基于人脸图像生成故事时，会因视觉线索对更具典型黑人特征和女性化特征的面孔施加更强的同质性偏见与刻板印象，并探讨其成因与缓解挑战。
摘要：
以GPT-4V为代表的视觉-语言模型（VLMs）能够熟练地整合文本与视觉两种模态。这种整合增强了大语言模型模拟人类感知的能力，使其能够处理图像输入。然而，尽管VLMs具有先进的能力，人们仍担忧VLMs会以某种方式继承两种模态的偏见，从而使偏见更加普遍且难以缓解。我们的研究探讨了VLMs如何在种族和性别方面延续同质性偏见和特质关联。当被要求根据人脸图像编写故事时，GPT-4V对从属种族和性别群体的描述比主导群体具有更高的同质性，并依赖于鲜明但总体正面的刻板印象。重要的是，VLM的刻板印象是由视觉线索驱动的，而不仅仅是群体身份本身，因此那些被评为更具典型黑人特征和女性化特征的面孔会受到更严重的刻板印象。这些发现表明，VLMs可能将与种族和性别群体相关的细微视觉线索与刻板印象关联起来，而这种关联方式可能难以缓解。我们探索了这一行为背后的深层原因，讨论了其影响，并强调了在VLMs逐渐趋近人类感知的过程中解决这些偏见的重要性。
英文摘要：
Vision Language Models (VLMs), exemplified by GPT-4V, adeptly integrate text and vision modalities. This integration enhances Large Language Models' ability to mimic human perception, allowing them to process image inputs. Despite VLMs' advanced capabilities, however, there is a concern that VLMs inherit biases of both modalities in ways that make biases more pervasive and difficult to mitigate. Our study explores how VLMs perpetuate homogeneity bias and trait associations with regards to race and gender. When prompted to write stories based on images of human faces, GPT-4V describes subordinate racial and gender groups with greater homogeneity than dominant groups and relies on distinct, yet generally positive, stereotypes. Importantly, VLM stereotyping is driven by visual cues rather than group membership alone such that faces that are rated as more prototypically Black and feminine are subject to greater stereotyping. These findings suggest that VLMs may associate subtle visual cues related to racial and gender groups with stereotypes in ways that could be challenging to mitigate. We explore the underlying reasons behind this behavior and discuss its implications and emphasize the importance of addressing these biases as VLMs come to mirror human perception.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
