标签混淆学习增强文本分类模型
Label Confusion Learning to Enhance Text Classification Models
摘要：
将真实标签表示为独热向量是训练文本分类模型的常见做法。然而，one-hot表示可能不能充分反映实例和标签之间的关系，因为标签通常不是完全独立的，并且实例在实践中可能涉及多个标签。不充分的one-hot表示往往会使模型训练过度自信，这可能导致任意预测和模型过拟合，特别是对于混乱的数据集（具有非常相似的标签的数据集）或噪声数据集（具有标签错误的数据集）。虽然使用标签平滑（LS）的训练模型可以在一定程度上缓解这个问题，但它仍然无法捕捉标签之间的真实关系。在本文中，我们提出了一种新的标签混淆模型（LCM）作为增强组件，目前流行的文本分类模型。LCM可以通过在训练过程中计算实例和标签之间的相似度来学习标签混淆以捕获标签之间的语义重叠，并生成更好的标签分布来替换原始的one-hot标签向量，从而提高最终的分类性能。在五个文本分类基准数据集上进行的大量实验揭示了LCM对几种广泛使用的深度学习分类模型的有效性。进一步的实验也验证了LCM方法对混乱或噪声数据集的处理特别有效，并且优于标签平滑方法。
英文摘要：
Representing a true label as a one-hot vector is a common practice in training text classification models. However, the one-hot representation may not adequately reflect the relation between the instances and labels, as labels are often not completely independent and instances may relate to multiple labels in practice. The inadequate one-hot representations tend to train the model to be over-confident, which may result in arbitrary prediction and model overfitting, especially for confused datasets (datasets with very similar labels) or noisy datasets (datasets with labeling errors). While training models with label smoothing (LS) can ease this problem in some degree, it still fails to capture the realistic relation among labels. In this paper, we propose a novel Label Confusion Model (LCM) as an enhancement component to current popular text classification models. LCM can learn label confusion to capture semantic overlap among labels by calculating the similarity between instances and labels during training and generate a better label distribution to replace the original one-hot label vector, thus improving the final classification performance. Extensive experiments on five text classification benchmark datasets reveal the effectiveness of LCM for several widely used deep learning classification models. Further experiments also verify that LCM is especially helpful for confused or noisy datasets and superior to the label smoothing method.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
