arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.07314cs.LGstat.ML
收藏

通过Logit调整的长尾学习

Long-tail learning via logit adjustment

Google Research(谷歌研究院)

Aditya Krishna Menon, Sadeep Jayasumana, Ankit Singh Rawat, Himanshu Jain, Andreas Veit, Sanjiv Kumar

摘要:

现实世界的分类问题通常表现出不平衡或长尾标签分布,其中许多标签仅与少数样本相关联。这对这些标签的泛化提出了挑战,也使朴素学习偏向于主导标签。在本文中,我们对标准softmax交叉熵训练进行了两个简单的修改,以应对这些挑战。我们的技术重新审视了基于标签频率的logit调整的经典思想,无论是事后应用于训练模型,还是在训练期间的损失中实施。这种调整鼓励在稀有标签与显性标签的logits之间存在较大的相对裕度。这些技术统一和概括了文献中最近的几个建议,同时拥有更坚实的统计基础和经验表现。

英文摘要:

Real-world classification problems typically exhibit an imbalanced or long-tailed label distribution, wherein many labels are associated with only a few samples. This poses a challenge for generalisation on such labels, and also makes naïve learning biased towards dominant labels. In this paper, we present two simple modifications of standard softmax cross-entropy training to cope with these challenges. Our techniques revisit the classic idea of logit adjustment based on the label frequencies, either applied post-hoc to a trained model, or enforced in the loss during training. Such adjustment encourages a large relative margin between logits of rare versus dominant labels. These techniques unify and generalise several recent proposals in the literature, while possessing firmer statistical grounding and empirical performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑