arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2409.08345cs.CVcs.LG
收藏

SIG:一种用于生成人脸识别评估数据集的合成身份生成流水线

SIG: A Synthetic Identity Generation Pipeline for Generating Evaluation Datasets for Face Recognition

Southern Methodist University(南卫理公会大学)

Kassi Nzalasse, Rishav Raj, Eli Laird, Corey Clark

AI总结:

针对人脸识别评估数据集伦理采集难、成本高的问题,提出可控制属性的合成身份生成流水线SIG,并发布均衡开源数据集ControlFace10k,验证了其用于算法评估与偏见分析的有效性。

摘要:

随着人工智能应用的拓展,模型评估面临更严格的审视。要确保模型可投入公共使用,需要用到评估数据集,这类数据集与训练数据的区别在于二者无交集,且需符合隐私法规、以符合伦理的方式采集。人脸识别系统的性能与公平性在很大程度上取决于这些评估数据集的质量与代表性。这类数据有时是在未经用户同意的情况下从网络抓取的,由此引发的伦理问题可能导致在未获得适当授权的情况下无法使用这些数据。在极少数情况下,数据是在受控环境中经同意后收集的,但这一过程耗时、成本高昂,且在执行层面存在后勤困难。这给那些无力承担采集符合伦理要求的评估数据集所需巨额资源的主体造成了障碍。为应对这些挑战,我们提出了合成身份生成流水线(Synthetic Identity Generation pipeline,简称SIG),可针对性创建符合伦理要求的均衡人脸识别评估数据集。我们提出并验证的该流水线能够生成高质量的合成身份人脸图像,且可控制姿态、面部特征以及种族、性别、年龄等人口统计属性。我们还发布了一个名为ControlFace10k的开源评估数据集,它包含3336个唯一合成身份的10008张人脸图像,在种族、性别和年龄维度上分布均衡,均通过所提出的SIG流水线生成。我们使用当前最先进的人脸识别算法,对ControlFace10k与非合成的BUPT数据集进行了分析,以验证其作为评估工具的有效性。该分析凸显了该数据集的特性,以及它在评估算法跨不同人口群体的偏见方面的实用价值。

英文摘要:

As Artificial Intelligence applications expand, the evaluation of models faces heightened scrutiny. Ensuring public readiness requires evaluation datasets, which differ from training data by being disjoint and ethically sourced in compliance with privacy regulations. The performance and fairness of face recognition systems depend significantly on the quality and representativeness of these evaluation datasets. This data is sometimes scraped from the internet without user's consent, causing ethical concerns that can prohibit its use without proper releases. In rare cases, data is collected in a controlled environment with consent, however, this process is time-consuming, expensive, and logistically difficult to execute. This creates a barrier for those unable to conjure the immense resources required to gather ethically sourced evaluation datasets. To address these challenges, we introduce the Synthetic Identity Generation pipeline, or SIG, that allows for the targeted creation of ethical, balanced datasets for face recognition evaluation. Our proposed and demonstrated pipeline generates high-quality images of synthetic identities with controllable pose, facial features, and demographic attributes, such as race, gender, and age. We also release an open-source evaluation dataset named ControlFace10k, consisting of 10,008 face images of 3,336 unique synthetic identities balanced across race, gender, and age, generated using the proposed SIG pipeline. We analyze ControlFace10k along with a non-synthetic BUPT dataset using state-of-the-art face recognition algorithms to demonstrate its effectiveness as an evaluation tool. This analysis highlights the dataset's characteristics and its utility in assessing algorithmic bias across different demographic groups.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑