基于显著性的多样性与公平性度量及FaceKeepOriginalAugment：一种增强公平性与多样性的新方法
Saliency-Based diversity and fairness Metric and FaceKeepOriginalAugment: A Novel Approach for Enhancing Fairness and Diversity
AI总结：
本研究提出数据增强方法FaceKeepOriginalAugment及基于显著性的多样性公平性度量，通过平衡数据多样性与信息保留缓解计算机视觉模型的多类偏差，经多数据集验证可有效降低性别偏差、提升公平性。
摘要：
数据增强已成为提升计算机视觉任务性能的关键工具，其中KeepOriginalAugment（保留原始增强）方法因能智能地将显著区域融入非显著区域、实现两类区域的增强而成为一项突出技术。尽管该方法在图像分类中表现出色，但其在解决偏差问题上的潜力尚未被探索。本研究中，我们提出KeepOriginalAugment方法的扩展版本，命名为FaceKeepOriginalAugment，用于探究计算机视觉模型中多方面的去偏问题——包括地域偏差、性别偏差和刻板印象偏差。通过在数据多样性与信息保留之间维持精妙平衡，我们的方法使模型能够同时利用多样化的显著区域和非显著区域，从而提升多样性并增强去偏效果。我们研究了多种确定显著区域放置位置的策略，以及通过交换视角来决定对哪部分进行增强的方法。借助Image Similarity Score（ISS，图像相似度评分），我们在多个数据集上量化了数据集多样性，包括Flickr Faces HQ（FFHQ）、WIKI、IMDB、Labelled Faces in the Wild（LFW）、UTK Faces和Diverse Dataset。我们在CEO、工程师、护士和教师数据集上评估了FaceKeepOriginalAugment缓解性别偏差的效果，在卷积神经网络（CNNs）和视觉Transformer（ViTs）中使用Image-Image Association Score（IIAS，图像-图像关联评分）进行评估。研究结果表明，FaceKeepOriginalAugment能有效提升计算机视觉模型的公平性与包容性，具体表现为性别偏差降低、整体公平性提升。此外，我们提出了一种新的度量指标——Saliency-Based Diversity and Fairness Metric（基于显著性的多样性与公平性度量），该指标可在处理各类数据集的数据不平衡问题的同时，量化多样性与公平性。
英文摘要：
Data augmentation has become a pivotal tool in enhancing the performance of computer vision tasks, with the KeepOriginalAugment method emerging as a standout technique for its intelligent incorporation of salient regions within less prominent areas, enabling augmentation in both regions. Despite its success in image classification, its potential in addressing biases remains unexplored. In this study, we introduce an extension of the KeepOriginalAugment method, termed FaceKeepOriginalAugment, which explores various debiasing aspects-geographical, gender, and stereotypical biases-in computer vision models. By maintaining a delicate balance between data diversity and information preservation, our approach empowers models to exploit both diverse salient and non-salient regions, thereby fostering increased diversity and debiasing effects. We investigate multiple strategies for determining the placement of the salient region and swapping perspectives to decide which part undergoes augmentation. Leveraging the Image Similarity Score (ISS), we quantify dataset diversity across a range of datasets, including Flickr Faces HQ (FFHQ), WIKI, IMDB, Labelled Faces in the Wild (LFW), UTK Faces, and Diverse Dataset. We evaluate the effectiveness of FaceKeepOriginalAugment in mitigating gender bias across CEO, Engineer, Nurse, and School Teacher datasets, utilizing the Image-Image Association Score (IIAS) in convolutional neural networks (CNNs) and vision transformers (ViTs). Our findings shows the efficacy of FaceKeepOriginalAugment in promoting fairness and inclusivity within computer vision models, demonstrated by reduced gender bias and enhanced overall fairness. Additionally, we introduce a novel metric, Saliency-Based Diversity and Fairness Metric, which quantifies both diversity and fairness while handling data imbalance across various datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
