打破偏见：论可泛化剪枝策略的局限性
Breaking Down Bias: On The Limits of Generalizable Pruning Strategies
AI总结：
本研究通过模型剪枝探究LLMs中种族偏见的概念化及可泛化缓解策略的可行性，发现剪枝可有效减偏但泛化性差，种族偏见部分通用、部分语境依赖，为AI法律责任分配提供参考。
摘要：
我们采用模型剪枝来研究大语言模型（LLMs）如何概念化种族偏见，以及针对此类偏见的可泛化缓解策略是否可行。我们的分析得出了几个新见解：剪枝可有效减少偏见且不会显著增加异常模型行为；基于神经元的剪枝策略通常比剪枝整个注意力头的方法效果更好。但研究结果也显示，随着剪枝策略更泛化，两种方法的有效性都会迅速下降，例如在金融决策场景中训练以消除种族偏见的模型，在商业交易场景中的偏见泛化效果很差。总体而言，我们的分析表明，种族偏见在语言模型中仅部分表现为通用概念，另一部分高度依赖具体语境，这意味着可泛化缓解策略的效果可能有限。我们的发现对AI相关法律框架具有重要意义，尤其表明有效的缓解策略应包括让在特定用例中部署模型的主体承担法律责任。
英文摘要：
We employ model pruning to examine how LLMs conceptualize racial biases, and whether a generalizable mitigation strategy for such biases appears feasible. Our analysis yields several novel insights. We find that pruning can be an effective method to reduce bias without significantly increasing anomalous model behavior. Neuron-based pruning strategies generally yield better results than approaches pruning entire attention heads. However, our results also show that the effectiveness of either approach quickly deteriorates as pruning strategies become more generalized. For instance, a model that is trained on removing racial biases in the context of financial decision-making poorly generalizes to biases in commercial transactions. Overall, our analysis suggests that racial biases are only partially represented as a general concept within language models. The other part of these biases is highly context-specific, suggesting that generalizable mitigation strategies may be of limited effectiveness. Our findings have important implications for legal frameworks surrounding AI. In particular, they suggest that an effective mitigation strategy should include the allocation of legal responsibility on those that deploy models in a specific use case.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
