基于自适应排序的弱监督类不平衡文本分类样本选择
Adaptive Ranking-based Sample Selection for Weakly Supervised Class-imbalanced Text Classification
摘要：
为了廉价地获得大量的训练标签，研究人员最近采用了弱监督（WS）范式，该范式利用标记规则来合成训练标签，而不是使用单个注释来实现自然语言处理（NLP）任务的竞争结果。然而，数据不平衡在应用WS范式时经常被忽视，尽管这是各种NLP任务中的常见问题。为了解决这一挑战，我们提出了自适应排名为基础的样本选择（ARS 2），一个模型无关的框架，以减轻数据不平衡的问题，在WS范式。具体而言，它根据当前模型的输出计算概率边际分数，以衡量和排名每个数据点的清洁度。然后，基于类和规则感知的排名对排名数据进行采样。特别是，这两个样本策略对应于我们的动机：（1）用平衡的数据批次训练模型，以减少数据不平衡问题;（2）利用每个标记规则的专业知识来收集干净的样本。在具有四种不同不平衡率的四个文本分类数据集上的实验表明，ARS 2的性能优于最先进的不平衡学习和WS方法，使其F1分数提高了2%-57.8%。
英文摘要：
To obtain a large amount of training labels inexpensively, researchers have recently adopted the weak supervision (WS) paradigm, which leverages labeling rules to synthesize training labels rather than using individual annotations to achieve competitive results for natural language processing (NLP) tasks. However, data imbalance is often overlooked in applying the WS paradigm, despite being a common issue in a variety of NLP tasks. To address this challenge, we propose Adaptive Ranking-based Sample Selection (ARS2), a model-agnostic framework to alleviate the data imbalance issue in the WS paradigm. Specifically, it calculates a probabilistic margin score based on the output of the current model to measure and rank the cleanliness of each data point. Then, the ranked data are sampled based on both class-wise and rule-aware ranking. In particular, the two sample strategies corresponds to our motivations: (1) to train the model with balanced data batches to reduce the data imbalance issue and (2) to exploit the expertise of each labeling rule for collecting clean samples. Experiments on four text classification datasets with four different imbalance ratios show that ARS2 outperformed the state-of-the-art imbalanced learning and WS methods, leading to a 2%-57.8% improvement on their F1-score.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
