当去偏措施适得其反:基于预处理的刻板印象缓解的反直觉副作用
When Debiasing Backfires: Counterintuitive Side Effects of Preprocessing-Based Stereotype Mitigation
AI总结:
研究基于预处理的刻板印象缓解方法的副作用,通过两个模型家族、多种策略及不同数据规模验证副作用存在,标准基准常忽略,注意力分析显示副作用与注意力流变化无关,还讨论评估意义并提供诊断方法及主张透明缓解措施。
摘要:
基于预处理的刻板印象缓解方法,如在去偏语料库上进行预/后训练,在自然语言处理中被广泛使用。虽然这些方法减少了目标群体中可测量的刻板印象,但研究发现它们常常引发意想不到的偏移副作用,即相对于其他人口统计特征的中性基线,刻板印象或反刻板印象可能会增加,包括跨不相关的人口统计类别。研究在两个模型家族、多种预处理策略以及不同数据规模的维基百科数据上进行预训练和后训练时展示了这些副作用。标准基准常常忽略这些偏移。通过注意力展开分析发现副作用并未伴随注意力流的大幅变化,这使得机理解释变得复杂。研究讨论了评估的意义,提供了可行的诊断方法,并主张采用具有副作用意识的透明缓解措施。
英文摘要:
Preprocessing-based methods for stereotype mitigation, such as pre-/post-training on debiased corpora, are widely used in NLP. While these approaches reduce measurable stereotypes for targeted groups, we find they often induce unintended shifts-side effects, where stereotyping or counter-stereotyping can increase relative to neutral baselines for other demographics, including across unrelated demographic categories. We demonstrate these side effects across two model families (encoder-only and decoder-only), multiple preprocessing strategies (removing stereotypical sentences, removing group mentions, and swapping group references), and both pre- and post-training at different data scales on Wikipedia. Standard benchmarks frequently miss these shifts. Using attention-rollout analysis, we observe that such side effects are not accompanied by large changes in attention flow, complicating mechanistic explanations. We discuss implications for evaluation, provide actionable diagnostics, and argue for side-effect-aware, transparent mitigation practices.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。