arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2109.07319cs.CLcs.AIcs.LG
收藏

InceptionXML:一种面向短文本极端分类的轻量级框架与同步负采样方法

InceptionXML: A Lightweight Framework with Synchronized Negative Sampling for Short Text Extreme Classification

Aalto University(阿尔托大学);Microsoft Corporation(微软公司);BITS Pilani(皮拉尼 Bits 大学);University of Bath(巴斯大学)

Siddhant Kharbanda, Atmadeep Banerjee, Devaansh Gupta, Akash Palrecha, Rohit Babbar

AI总结:

本文提出轻量级卷积架构InceptionXML和同步负采样流水线SyncXML,用于短文本极端分类,在模型大小和推理效率上显著优于现有方法,仅需2% FLOPs即可超越Transformer基线。

摘要:

将短文本数据自动标注到大量目标标签上,即短文本极端分类,已在相关搜索预测和产品推荐等众多应用中得到应用。本文提出了一种卷积架构InceptionXML,该架构轻量但功能强大,并且对搜索和推荐中短文本查询固有的词序缺失具有鲁棒性。我们通过将卷积操作沿嵌入维度而非传统文本分类CNN中沿词维度重新构造,证明了应用卷积的有效性。为了将模型扩展到具有数百万标签的数据集,我们还提出了SyncXML流水线,它通过同步标签短列表器和极端分类器,改进了最近提出的用于标签短列表的动态硬负样本挖掘技术的不足。SyncXML不仅将推理时间减少到一半,而且在模型大小上比最先进的Astec小一个数量级。通过全面的实证比较,我们表明InceptionXML不仅在基准数据集上优于现有方法,而且优于仅需2% FLOPs的Transformer基线。InceptionXML的代码可在https://github.com/xmc-aalto/inceptionxml获取。

英文摘要:

Automatic annotation of short-text data to a large number of target labels, referred to as Short Text Extreme Classification, has found numerous applications including prediction of related searches and product recommendation. In this paper, we propose a convolutional architecture InceptionXML which is light-weight, yet powerful, and robust to the inherent lack of word-order in short-text queries encountered in search and recommendation. We demonstrate the efficacy of applying convolutions by recasting the operation along the embedding dimension instead of the word dimension as applied in conventional CNNs for text classification. Towards scaling our model to datasets with millions of labels, we also propose SyncXML pipeline which improves upon the shortcomings of the recently proposed dynamic hard-negative mining technique for label short-listing by synchronizing the label-shortlister and extreme classifier. SyncXML not only reduces the inference time to half but is also an order of magnitude smaller than state-of-the-art Astec in terms of model size. Through a comprehensive empirical comparison, we show that not only can InceptionXML outperform existing approaches on benchmark datasets but also the transformer baselines requiring only 2% FLOPs. The code for InceptionXML is available at https://github.com/xmc-aalto/inceptionxml.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑