arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2010.01653cs.CL
收藏

包含少量和Zero-Shot标签的大规模多标签文本分类的实验研究

An Empirical Study on Large-Scale Multi-Label Text Classification Including Few and Zero-Shot Labels

Athens University of Economics and Business(雅典经济与商业大学);NCSR “Demokritos”(德谟克利特国家科学研究中心);University of Sheffield(谢菲尔德大学)

Ilias Chalkidis, Manos Fergadiotis, Sotiris Kotitsas, Prodromos Malakasiotis, Nikolaos Aletras, Ion Androutsopoulos

摘要:

大规模多标签文本分类(LMTC)在自然语言处理(NLP)领域有着广泛的应用,同时也带来了一些有趣的挑战。首先,由于LMTC数据集的标签集非常大,并且标签分布不均匀,因此并非所有标签都能在训练集中得到很好的表示。此外,标签层次结构和人类标签指南的差异可能会影响图形感知的注释接近度。最后,标签层次结构定期更新,需要能够zero-shot泛化的LMTC模型。当前最先进的LMTC模型采用标签式注意力网络(LWAN),其(1)通常将LMTC视为平面多标签分类;(2)可以使用标签层次结构来改进零样本学习,尽管这种做法在很大程度上未得到充分研究;(3)尚未与预训练的Transformers(例如BERT)相结合,这已经在几个NLP基准测试中产生了最先进的结果。在这里,我们第一次经验性地评估了一系列LMTC方法,从香草LWAN到分层分类方法和迁移学习,在来自不同领域的三个数据集上进行频繁,少量和零样本学习。我们表明,基于概率标签树(PLT)的分层方法优于LWAN。此外,我们表明,基于变换器的方法优于国家的最先进的两个数据集,我们提出了一个新的国家的最先进的方法,结合BERT与LWAN。最后,我们提出了新的模型,利用标签层次结构,以改善少数和零样本学习,考虑在每个数据集上的图形感知的注释接近度的措施,我们介绍。

英文摘要:

Large-scale Multi-label Text Classification (LMTC) has a wide range of Natural Language Processing (NLP) applications and presents interesting challenges. First, not all labels are well represented in the training set, due to the very large label set and the skewed label distributions of LMTC datasets. Also, label hierarchies and differences in human labelling guidelines may affect graph-aware annotation proximity. Finally, the label hierarchies are periodically updated, requiring LMTC models capable of zero-shot generalization. Current state-of-the-art LMTC models employ Label-Wise Attention Networks (LWANs), which (1) typically treat LMTC as flat multi-label classification; (2) may use the label hierarchy to improve zero-shot learning, although this practice is vastly understudied; and (3) have not been combined with pre-trained Transformers (e.g. BERT), which have led to state-of-the-art results in several NLP benchmarks. Here, for the first time, we empirically evaluate a battery of LMTC methods from vanilla LWANs to hierarchical classification approaches and transfer learning, on frequent, few, and zero-shot learning on three datasets from different domains. We show that hierarchical methods based on Probabilistic Label Trees (PLTs) outperform LWANs. Furthermore, we show that Transformer-based approaches outperform the state-of-the-art in two of the datasets, and we propose a new state-of-the-art method which combines BERT with LWANs. Finally, we propose new models that leverage the label hierarchy to improve few and zero-shot learning, considering on each dataset a graph-aware annotation proximity measure that we introduce.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑