SelfMix：自混淆训练的文本标签噪声鲁棒学习
SelfMix: Robust Learning Against Textual Label Noise with Self-Mixup Training
摘要：
文本分类的传统成功依赖于注释数据，而预训练语言模型（PLM）的新范式仍然需要一些标记数据用于下游任务。然而，在实际应用中，训练数据中不可避免地存在标签噪声，从而损害了基于此类数据构建的模型的有效性、鲁棒性和泛化性。近年来，人们在视觉数据的处理上取得了显著的成就，而在文本数据的处理上却鲜有研究。为了填补这一空白，我们提出了SelfMix，一种简单而有效的方法，用于处理文本分类任务中的标签噪声。SelfMix使用高斯混合模型来分离样本，并利用半监督学习。与以前需要多个模型的工作不同，我们的方法利用单个模型上的辍学机制来减少自我训练中的确认偏差，并引入了文本级混合训练策略。在三个具有不同类型文本的文本分类基准上的实验结果表明，在不同的噪声比和噪声类型下，我们所提出的方法的性能优于这些针对文本和视觉数据设计的强基线。我们的代码可在https://github.com/noise-learning/SelfMix上获得。
英文摘要：
The conventional success of textual classification relies on annotated data, and the new paradigm of pre-trained language models (PLMs) still requires a few labeled data for downstream tasks. However, in real-world applications, label noise inevitably exists in training data, damaging the effectiveness, robustness, and generalization of the models constructed on such data. Recently, remarkable achievements have been made to mitigate this dilemma in visual data, while only a few explore textual data. To fill this gap, we present SelfMix, a simple yet effective method, to handle label noise in text classification tasks. SelfMix uses the Gaussian Mixture Model to separate samples and leverages semi-supervised learning. Unlike previous works requiring multiple models, our method utilizes the dropout mechanism on a single model to reduce the confirmation bias in self-training and introduces a textual-level mixup training strategy. Experimental results on three text classification benchmarks with different types of text show that the performance of our proposed method outperforms these strong baselines designed for both textual and visual data under different noise ratios and noise types. Our code is available at https://github.com/noise-learning/SelfMix.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
