arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.19130cs.LGcs.AI
收藏

利用影响函数检测标签偏差

Detecting labeling bias using influence functions

Copenhagen University(哥本哈根大学);Technical University of Denmark(丹麦技术大学)

Frida Jørgensen, Nina Weng, Siavash Bigdeli

AI总结:

本文利用影响函数检测标签偏差,通过分析训练样本对模型预测的影响,识别错误标签样本,验证了影响函数在检测标签偏差中的有效性。

摘要:

标签偏差在数据收集过程中由于资源限制或无意识偏见产生,导致不同子群体的标签错误率不均或子群体 prevalence 的误表征。大多数公平性约束假设训练标签反映真实分布,当存在标签偏差时,这些约束变得无效;留下一个有挑战性的问题,即如何检测此类标签偏差?在本文中,我们研究影响函数是否可用于检测标签偏差。影响函数通过利用损失函数的梯度和Hessian来估计每个训练样本对模型预测的影响——当出现标签错误时,影响函数可以识别训练集中的错误标签样本,揭示底层的失败模式。我们开发了一个样本估值流程,并首先在MNIST数据集上测试,然后扩展到更复杂的CheXpert医学影像数据集。为了检验标签噪声,我们通过翻转数据集中某一类别的20%标签引入受控错误。使用对角Hessian近似,我们展示了有希望的结果,成功在MNIST中检测到近90%的错误标签样本。在CheXpert中,错误标签样本始终表现出更高的影响分数。这些结果突显了影响函数在识别标签错误方面的潜力。

英文摘要:

Labeling bias arises during data collection due to resource limitations or unconscious bias, leading to unequal label error rates across subgroups or misrepresentation of subgroup prevalence. Most fairness constraints assume training labels reflect the true distribution, rendering them ineffective when labeling bias is present; leaving a challenging question, that \textit{how can we detect such labeling bias?} In this work, we investigate whether influence functions can be used to detect labeling bias. Influence functions estimate how much each training sample affects a model's predictions by leveraging the gradient and Hessian of the loss function -- when labeling errors occur, influence functions can identify wrongly labeled samples in the training set, revealing the underlying failure mode. We develop a sample valuation pipeline and test it first on the MNIST dataset, then scaled to the more complex CheXpert medical imaging dataset. To examine label noise, we introduced controlled errors by flipping 20\% of the labels for one class in the dataset. Using a diagonal Hessian approximation, we demonstrated promising results, successfully detecting nearly 90\% of mislabeled samples in MNIST. On CheXpert, mislabeled samples consistently exhibit higher influence scores. These results highlight the potential of influence functions for identifying label errors.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑