人脸识别中算法偏见的基准评测：一种使用合成人脸与人类评估的实验方法
Benchmarking Algorithmic Bias in Face Recognition: An Experimental Approach Using Synthetic Faces and Human Evaluation
AI总结：
本文提出一种基于神经生成合成人脸和人类评估的实验方法，通过独立操纵属性来因果性测量人脸识别系统的种族与性别偏见，并验证了三个研究级模型对黑人和东亚人群准确率较低。
摘要：
我们提出了一种用于测量人脸识别系统中偏见的实验方法。现有的偏见测量方法依赖于在自然场景中收集并标注了受保护属性（如种族、性别）和非受保护属性（如姿态、光照）的基准数据集。此类观测性数据集只能得出相关性结论，例如“算法A在数据集X中女性和男性人脸上的准确率不同”。相比之下，实验方法通过单独操纵各个属性，从而能够得出因果性结论，例如“算法A的准确率受到性别和肤色的影响”。我们的方法基于使用神经人脸生成器生成合成人脸，其中每个感兴趣的属性被独立修改，同时保持所有其他属性不变。人类观察者关键性地提供了合成图像对之间感知身份相似性的真实标注。我们通过评估三个研究级人脸识别模型的种族和性别偏见，定量验证了我们的方法。我们的合成流程揭示，对于这些算法，黑人和东亚人群子组的准确率较低。我们的方法还可以量化属性的感知变化如何影响这些模型所报告的人脸身份距离。我们的大型合成数据集包含48,000对合成人脸图像（10,200张唯一合成人脸）和555,000条人类标注（个体属性及成对身份比较），可供这一重要领域的研究人员使用。
英文摘要：
We propose an experimental method for measuring bias in face recognition systems. Existing methods to measure bias depend on benchmark datasets that are collected in the wild and annotated for protected (e.g., race, gender) and non-protected (e.g., pose, lighting) attributes. Such observational datasets only permit correlational conclusions, e.g., "Algorithm A's accuracy is different on female and male faces in dataset X.". By contrast, experimental methods manipulate attributes individually and thus permit causal conclusions, e.g., "Algorithm A's accuracy is affected by gender and skin color." Our method is based on generating synthetic faces using a neural face generator, where each attribute of interest is modified independently while leaving all other attributes constant. Human observers crucially provide the ground truth on perceptual identity similarity between synthetic image pairs. We validate our method quantitatively by evaluating race and gender biases of three research-grade face recognition models. Our synthetic pipeline reveals that for these algorithms, accuracy is lower for Black and East Asian population subgroups. Our method can also quantify how perceptual changes in attributes affect face identity distances reported by these models. Our large synthetic dataset, consisting of 48,000 synthetic face image pairs (10,200 unique synthetic faces) and 555,000 human annotations (individual attributes and pairwise identity comparisons) is available to researchers in this important area.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
