表征代理校准的不确定性来源，消除注释器和数据偏差的歧义
Characterizing Sources of Uncertainty to Proxy Calibration and Disambiguate Annotator and Data Bias
摘要：
对于注释者可以合理地不同意的复杂现象（例如情感识别），支持模型可解释性是一项具有挑战性的机器学习任务。在这项工作中，我们表明，明确量化的不确定性，在这样的设置具有可解释性的好处。我们使用一个简单的修改的经典网络推理，使用蒙特卡罗辍学给措施的认识和任意的不确定性。我们确定了一个显着的相关性任意的不确定性和人类注释者的分歧（$r\approx.3$）。此外，我们还展示了如何使用任意不确定性来识别困难和主观的训练样本，以及认识不确定性如何揭示可能导致不公平预测的数据偏差。我们将总不确定性确定为模型校准的合适替代品，即我们可以信任模型预测置信度的程度。除了可解释性的好处，我们观察到适度的性能提升纳入模型的不确定性。
英文摘要：
Supporting model interpretability for complex phenomena where annotators can legitimately disagree, such as emotion recognition, is a challenging machine learning task. In this work, we show that explicitly quantifying the uncertainty in such settings has interpretability benefits. We use a simple modification of a classical network inference using Monte Carlo dropout to give measures of epistemic and aleatoric uncertainty. We identify a significant correlation between aleatoric uncertainty and human annotator disagreement ($r\approx.3$). Additionally, we demonstrate how difficult and subjective training samples can be identified using aleatoric uncertainty and how epistemic uncertainty can reveal data bias that could result in unfair predictions. We identify the total uncertainty as a suitable surrogate for model calibration, i.e. the degree we can trust model's predicted confidence. In addition to explainability benefits, we observe modest performance boosts from incorporating model uncertainty.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
