arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2005.08469cs.CL
收藏

基于控制泛化的少样本文本分类

Text Classification with Few Examples using Controlled Generalization

Google(谷歌);Pinterest;UPenn(宾夕法尼亚大学)

Abhijit Mahabal, Jason Baldridge, Burcu Karagol Ayan, Vincent Perot, Dan Roth

摘要:

用于文本分类的训练数据在实际应用中往往是有限的,特别是对于具有许多输出类或涉及许多相关分类问题的应用。这意味着分类器必须从有限的证据中进行归纳,但归纳的方式和程度取决于任务。目前的实践主要依赖于预训练的词嵌入来将训练中未看到的词映射到类似的已看到的词。不幸的是,这将意义的许多组成部分压缩成高度有限的能力。我们的替代方案从未标记的解析语料库中获得的稀疏预训练表示开始;基于可用的训练数据,我们选择提供相关概括的特征。这产生了特定于任务的语义向量;在这里,我们表明,与现有的最先进的方法相比,这些向量上的前馈网络在低数据场景中特别有效。通过将该网络与卷积神经网络进一步配对,我们在低数据场景中保持了这种优势,并在使用完整训练集时保持竞争力。

英文摘要:

Training data for text classification is often limited in practice, especially for applications with many output classes or involving many related classification problems. This means classifiers must generalize from limited evidence, but the manner and extent of generalization is task dependent. Current practice primarily relies on pre-trained word embeddings to map words unseen in training to similar seen ones. Unfortunately, this squishes many components of meaning into highly restricted capacity. Our alternative begins with sparse pre-trained representations derived from unlabeled parsed corpora; based on the available training data, we select features that offers the relevant generalizations. This produces task-specific semantic vectors; here, we show that a feed-forward network over these vectors is especially effective in low-data scenarios, compared to existing state-of-the-art methods. By further pairing this network with a convolutional neural network, we keep this edge in low data scenarios and remain competitive when using full training sets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑