arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2006.08564cs.LGstat.ML
收藏

消除神经网络偏置的帧内处理方法

Intra-Processing Methods for Debiasing Neural Networks

RAIR Lab RPI(伦斯勒理工学院RAIR实验室)

Yash Savani, Colin White, Naveen Sundar Govindarajulu

摘要:

随着深度学习模型越来越多地承担影响人类生活的决策任务,例如犯罪累犯、贷款偿还和执法人脸识别,偏见正成为一个越来越令人担忧的问题。去偏置算法通常分为三种范式:预处理、处理中和后处理。然而,在计算机视觉或自然语言应用中,通常从大型通用模型开始,然后根据特定用例进行微调。预处理或处理中的方法需要从头开始重新训练整个模型,而后处理方法只能对模型进行黑盒访问,因此它们不会利用训练模型的权重。专门为这种微调用例创建去偏置算法在很大程度上被忽视了。在这项工作中,我们发起了一个新的范式去偏置研究,内处理,它位于处理和后处理方法之间的研究。内部处理方法是专门设计来消除大型模型的偏差的,这些模型是在通用数据集上训练的,并在更具体的任务上进行了微调。我们展示了如何针对这种用例重新利用现有的处理方法,我们还提出了三种基线算法:随机扰动,分层优化和对抗性微调。我们所有的技术都可以用于所有流行的群体公平性度量,如均衡赔率或统计奇偶性差异。我们在AIF360工具包中的三个流行数据集以及CelebA人脸数据集上评估了这些方法。我们的代码可在https://github.com/abacusai/intraprocessing_debiasing上获得。

英文摘要:

As deep learning models become tasked with more and more decisions that impact human lives, such as criminal recidivism, loan repayment, and face recognition for law enforcement, bias is becoming a growing concern. Debiasing algorithms are typically split into three paradigms: pre-processing, in-processing, and post-processing. However, in computer vision or natural language applications, it is common to start with a large generic model and then fine-tune to a specific use-case. Pre- or in-processing methods would require retraining the entire model from scratch, while post-processing methods only have black-box access to the model, so they do not leverage the weights of the trained model. Creating debiasing algorithms specifically for this fine-tuning use-case has largely been neglected. In this work, we initiate the study of a new paradigm in debiasing research, intra-processing, which sits between in-processing and post-processing methods. Intra-processing methods are designed specifically to debias large models which have been trained on a generic dataset and fine-tuned on a more specific task. We show how to repurpose existing in-processing methods for this use-case, and we also propose three baseline algorithms: random perturbation, layerwise optimization, and adversarial fine-tuning. All of our techniques can be used for all popular group fairness measures such as equalized odds or statistical parity difference. We evaluate these methods across three popular datasets from the AIF360 toolkit, as well as on the CelebA faces dataset. Our code is available at https://github.com/abacusai/intraprocessing_debiasing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑