行动前三思：一种缓解视觉语言任务中性别偏见的两阶段框架
Think Before You Act: A Two-Stage Framework for Mitigating Gender Bias Towards Vision-Language Tasks
AI总结：
针对视觉语言模型中的性别偏见问题，提出任务无关的两阶段框架GAMA，通过生成性别模糊叙事并整合图像与问题提示进行答案推断，有效缓解偏见并展现良好泛化能力。
摘要：
视觉语言模型（VLM）中的性别偏见可能强化有害的刻板印象和歧视。本文聚焦于缓解视觉语言任务中的性别偏见。我们识别出物体幻觉是VLM中性别偏见的本质。现有的VLM倾向于关注图像中显著或熟悉的属性，而忽略情境化的细微差别。此外，大多数VLM依赖特定物体与性别属性之间的共现关系来推断被忽略的特征，最终导致性别偏见。我们提出GAMA，一个任务无关的生成框架，用于缓解性别偏见。GAMA包含两个阶段：叙事生成和答案推断。在叙事生成阶段，GAMA生成全面但性别模糊的叙事，防止过早集中于局部图像特征，尤其是性别属性。在答案推断阶段，GAMA整合图像、生成的叙事和任务特定的问题提示，为不同的视觉语言任务推断答案。这种方法使模型能够重新思考性别属性和答案。我们对GAMA进行了大量实验，证明了其去偏见能力和泛化能力。
英文摘要：
Gender bias in vision-language models (VLMs) can reinforce harmful stereotypes and discrimination. In this paper, we focus on mitigating gender bias towards vision-language tasks. We identify object hallucination as the essence of gender bias in VLMs. Existing VLMs tend to focus on salient or familiar attributes in images but ignore contextualized nuances. Moreover, most VLMs rely on the co-occurrence between specific objects and gender attributes to infer the ignored features, ultimately resulting in gender bias. We propose GAMA, a task-agnostic generation framework to mitigate gender bias. GAMA consists of two stages: narrative generation and answer inference. During narrative generation, GAMA yields all-sided but gender-obfuscated narratives, which prevents premature concentration on localized image features, especially gender attributes. During answer inference, GAMA integrates the image, generated narrative, and a task-specific question prompt to infer answers for different vision-language tasks. This approach allows the model to rethink gender attributes and answers. We conduct extensive experiments on GAMA, demonstrating its debiasing and generalization ability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
