基于AdapterFusion的参数高效模块化偏见缓解
Parameter-efficient Modularised Bias Mitigation via AdapterFusion
AI总结：
本文提出DAM方法，利用AdapterFusion将偏见缓解功能封装为独立适配器，按需集成到预训练语言模型中，实现参数高效的模块化去偏，同时保持核心模型不变，实验表明其在多个任务上有效缓解偏见并避免灾难性遗忘。
摘要：
大型预训练语言模型包含社会偏见，并将这些偏见带到下游任务中。当前的进程内偏见缓解方法（如对抗训练）通过更新模型参数来强制去偏，实际上将模型转移到一个新的、不可逆的去偏状态。在这项工作中，我们提出了一种新颖的方法，开发独立于模型的去偏功能，这些功能可以按需集成到模型中，同时保持核心模型不变。借鉴多任务学习中的AdapterFusion概念，我们引入了DAM（使用适配器模块去偏）——一种去偏方法，首先将任意偏见缓解功能封装到单独的适配器中，然后按需将它们添加到模型中，以提供公平性。我们在三个分类任务上进行了大量实验，以性别、种族和年龄作为受保护属性。我们的结果表明，DAM提高或保持了偏见缓解的有效性，避免了多属性场景中的灾难性遗忘，并保持了相当的任务性能，同时实现了参数效率和原始模型与去偏模型之间的轻松切换。
英文摘要：
Large pre-trained language models contain societal biases and carry along these biases to downstream tasks. Current in-processing bias mitigation approaches (like adversarial training) impose debiasing by updating a model's parameters, effectively transferring the model to a new, irreversible debiased state. In this work, we propose a novel approach to develop stand-alone debiasing functionalities separate from the model, which can be integrated into the model on-demand, while keeping the core model untouched. Drawing from the concept of AdapterFusion in multi-task learning, we introduce DAM (Debiasing with Adapter Modules) - a debiasing approach to first encapsulate arbitrary bias mitigation functionalities into separate adapters, and then add them to the model on-demand in order to deliver fairness qualities. We conduct a large set of experiments on three classification tasks with gender, race, and age as protected attributes. Our results show that DAM improves or maintains the effectiveness of bias mitigation, avoids catastrophic forgetting in a multi-attribute scenario, and maintains on-par task performance, while granting parameter-efficiency and easy switching between the original and debiased models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
