神经网络反对（与支持）自训练：基于少量标注与大量未标注集的分类
Neural Networks Against (and For) Self-Training: Classification with Small Labeled and Large Unlabeled Sets
AI总结：
提出基于自训练的半监督文本分类器，通过重塑伪标签层级结构和引入基于子采样预测不确定性的混合度量，解决了语义漂移和置信度校准差的问题，在五个基准上显著优于十个基线模型且可与语言模型预训练叠加。
摘要：
我们提出了一种基于自训练的半监督文本分类器，利用了神经网络的一个正面和一个负面特性。自训练的一个弱点是语义漂移问题，即噪声伪标签在迭代中不断累积，导致错误率剧增。为了应对这一挑战，我们重新塑造了伪标签的角色，并创建了信息的层级结构。此外，自训练中的一个关键步骤是利用分类器的置信度预测来选择最佳的候选伪标签。神经网络无法高效地完成这一步骤，因为众所周知其输出的校准性很差。为了克服这一挑战，我们提出了一种混合度量来取代单纯的置信度测量。我们的度量通过子采样技术考虑了预测的不确定性。我们在一组五个标准基准上评估了我们的模型，并表明它显著优于一组十个不同的基线模型。此外，我们展示了我们模型取得的改进可与语言模型预训练相叠加，而后者是一种广泛使用的利用未标注文档的技术。我们的代码可在 https://github.com/p-karisani/RST 获取。
英文摘要：
We propose a semi-supervised text classifier based on self-training using one positive and one negative property of neural networks. One of the weaknesses of self-training is the semantic drift problem, where noisy pseudo-labels accumulate over iterations and consequently the error rate soars. In order to tackle this challenge, we reshape the role of pseudo-labels and create a hierarchical order of information. In addition, a crucial step in self-training is to use the classifier confidence prediction to select the best candidate pseudo-labels. This step cannot be efficiently done by neural networks, because it is known that their output is poorly calibrated. To overcome this challenge, we propose a hybrid metric to replace the plain confidence measurement. Our metric takes into account the prediction uncertainty via a subsampling technique. We evaluate our model in a set of five standard benchmarks, and show that it significantly outperforms a set of ten diverse baseline models. Furthermore, we show that the improvement achieved by our model is additive to language model pretraining, which is a widely used technique for using unlabeled documents. Our code is available at https://github.com/p-karisani/RST.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
