IDoFew：面向少标签文本分类的语言模型基于双聚类的中间训练
IDoFew: Intermediate Training Using Dual-Clustering in Language Models for Few Labels Text Classification
AI总结：
针对少标签文本分类的冷启动问题，提出两阶段中间聚类结合微调的IDoFew模型，利用互补算法生成可靠伪标签以减少预测错误，显著优于对比模型。
摘要：
诸如来自 Transformers 的双向编码器表示（BERT）等语言模型在包括文本分类在内的各种自然语言处理（NLP）和文本挖掘任务中一直非常有效。然而，一些任务对这些模型仍然构成挑战，包括标签有限的文本分类。这可能导致冷启动问题。尽管一些方法试图通过将单阶段聚类作为中间训练步骤与预训练语言模型相结合来解决这一问题，从而生成伪标签以改进分类，但由于聚类算法的局限性，这些方法往往容易出错。为了克服这一点，我们开发了一种新颖的两阶段中间聚类并随后进行微调的方法，该方法能够可靠地对伪标签进行建模，从而减少了预测错误。我们的模型 IDoFew 的关键创新在于，两阶段聚类结合两种不同的聚类算法有助于发挥互补算法的优势，从而减少在生成用于微调的可靠伪标签时的错误。与强大的对比模型相比，我们的方法显示出了显著的改进。
英文摘要：
Language models such as Bidirectional Encoder Representations from Transformers (BERT) have been very effective in various Natural Language Processing (NLP) and text mining tasks including text classification. However, some tasks still pose challenges for these models, including text classification with limited labels. This can result in a cold-start problem. Although some approaches have attempted to address this problem through single-stage clustering as an intermediate training step coupled with a pre-trained language model, which generates pseudo-labels to improve classification, these methods are often error-prone due to the limitations of the clustering algorithms. To overcome this, we have developed a novel two-stage intermediate clustering with subsequent fine-tuning that models the pseudo-labels reliably, resulting in reduced prediction errors. The key novelty in our model, IDoFew, is that the two-stage clustering coupled with two different clustering algorithms helps exploit the advantages of the complementary algorithms that reduce the errors in generating reliable pseudo-labels for fine-tuning. Our approach has shown significant improvements compared to strong comparative models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
