arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2407.08189cs.CLcs.AI
收藏

fairBERTs:通过语义与公平感知扰动擦除敏感信息

fairBERTs: Erasing Sensitive Information Through Semantic and Fairness-aware Perturbations

Alibaba Group(阿里巴巴集团)

Jinfeng Li, Yuefeng Chen, Xiangyu Liu, Longtao Huang, Rong Zhang, Hui Xue

AI总结:

针对预训练语言模型存在刻板偏见的公平性问题,提出fairBERTs框架,借助生成对抗网络的语义与公平感知扰动擦除敏感信息,在保持效用的同时缓解不公平,且对抗组件可迁移提升其他类BERT模型的公平性。

摘要:

预训练语言模型(PLMs)革新了自然语言处理的研究与应用。然而,PLMs中编码的刻板偏见(如性别和种族歧视)带来了负面伦理问题,严重限制了其更广泛的应用。为解决上述不公平问题,我们提出fairBERTs,这是一个学习公平微调BERT系列模型的通用框架,通过生成对抗网络生成的语义与公平感知扰动擦除受保护的敏感信息。通过在两个真实世界任务上开展大量定性与定量实验,我们证明fairBERTs在缓解不公平性的同时能保持模型效用,具有显著优势。我们还验证了将fairBERTs中的对抗组件迁移到其他常规训练的类BERT模型以提升公平性的可行性。我们的发现可为构建更公平的微调PLMs的后续研究提供启发。

英文摘要:

Pre-trained language models (PLMs) have revolutionized both the natural language processing research and applications. However, stereotypical biases (e.g., gender and racial discrimination) encoded in PLMs have raised negative ethical implications for PLMs, which critically limits their broader applications. To address the aforementioned unfairness issues, we present fairBERTs, a general framework for learning fair fine-tuned BERT series models by erasing the protected sensitive information via semantic and fairness-aware perturbations generated by a generative adversarial network. Through extensive qualitative and quantitative experiments on two real-world tasks, we demonstrate the great superiority of fairBERTs in mitigating unfairness while maintaining the model utility. We also verify the feasibility of transferring adversarial components in fairBERTs to other conventionally trained BERT-like models for yielding fairness improvements. Our findings may shed light on further research on building fairer fine-tuned PLMs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑