FairDistillation：减轻语言模型中的定型
FairDistillation: Mitigating Stereotyping in Language Models
摘要：
大型预训练的语言模型正成功地用于多种语言的各种任务中。随着这种使用的不断增加，有害副作用的风险也在增加，例如通过复制和加强陈规定型观念。然而，检测和减轻这些危害通常很难做到，并且在处理多种语言或考虑不同偏见时会变得计算昂贵。为了解决这个问题，我们提出了FairDistillation：一种基于知识蒸馏的跨语言方法，在控制特定偏差的同时构建较小的语言模型。我们发现，我们的蒸馏方法不会对大多数任务的下游性能产生负面影响，并成功地减轻了刻板印象和代表性伤害。我们证明，FairDistillation可以创建更公平的语言模型在一个相当低的成本比其他方法。
英文摘要：
Large pre-trained language models are successfully being used in a variety of tasks, across many languages. With this ever-increasing usage, the risk of harmful side effects also rises, for example by reproducing and reinforcing stereotypes. However, detecting and mitigating these harms is difficult to do in general and becomes computationally expensive when tackling multiple languages or when considering different biases. To address this, we present FairDistillation: a cross-lingual method based on knowledge distillation to construct smaller language models while controlling for specific biases. We found that our distillation method does not negatively affect the downstream performance on most tasks and successfully mitigates stereotyping and representational harms. We demonstrate that FairDistillation can create fairer language models at a considerably lower cost than alternative approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
