说出我的名字:模型偏差发现框架
Say My Name: a Model's Bias Discovery Framework
AI总结:
提出SaMyNa框架,通过基于文本的语义流程识别深度模型习得的偏差,增强可解释性并支持训练或事后验证中的去偏差,在传统基准上验证了有效性。
摘要:
在过去几年中,由于深度学习在下游任务中的广泛适用性以及端到端的训练能力,人们越来越关注模型对特定、非代表性模式的潜在偏差问题。许多专注于无监督去偏差的研究通常利用深度模型倾向于学习“更简单”样本的特性,例如通过对潜在空间进行聚类来获取偏差伪标签。然而,这类伪标签的解释并不简单,尤其是对于非专业终端用户而言,因为它无法提供关于偏差特征的语义信息。为解决这一问题,我们提出了“说出我的名字”(SaMyNa),这是首个能够在语义层面识别深度模型内部偏差的工具。与现有方法不同,我们的方法专注于模型已习得的偏差。我们的基于文本的流程增强了可解释性,并支持去偏差工作:该方法既可在训练期间应用,也可用于事后验证,能够解耦与任务相关的信息,并作为分析偏差的工具。在传统基准上的评估证明了其在检测偏差甚至否定偏差方面的有效性,展示了其在模型诊断中的广泛适用性。
英文摘要:
In the last few years, due to the broad applicability of deep learning to downstream tasks and end-to-end training capabilities, increasingly more concerns about potential biases to specific, non-representative patterns have been raised. Many works focusing on unsupervised debiasing usually leverage the tendency of deep models to learn ``easier'' samples, for example by clustering the latent space to obtain bias pseudo-labels. However, the interpretation of such pseudo-labels is not trivial, especially for a non-expert end user, as it does not provide semantic information about the bias features. To address this issue, we introduce ``Say My Name'' (SaMyNa), the first tool to identify biases within deep models semantically. Unlike existing methods, our approach focuses on biases learned by the model. Our text-based pipeline enhances explainability and supports debiasing efforts: applicable during either training or post-hoc validation, our method can disentangle task-related information and proposes itself as a tool to analyze biases. Evaluation on traditional benchmarks demonstrates its effectiveness in detecting biases and even disclaiming them, showcasing its broad applicability for model diagnosis.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。