arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.01638cs.CLcs.AI
收藏

用于高效且可解释的长文本分类的多维感知机

Multidimensional Perceptron for Efficient and Explainable Long Text Classification

Alibaba Group(阿里巴巴集团)

Yexiang Wang, Yating Zhang, Xiaozhong Liu, Changlong Sun

AI总结:

提出段感知多维感知机(SWIPE)替代注意力/RNN,通过监督学习整体标签与无监督感知分段贡献,在提升长文本分类效率与准确率的同时实现优越可解释性。

摘要:

由于 transformer 和预训练模型不可避免的成本与复杂性,长文本分类引发了效率担忧。同时,在医疗和法律等高度敏感的长文本挖掘领域,潜在的模型不信任问题虽被低估且缺乏探索,却可能引发严重担忧。现有方法通常将长文本分段,用预训练模型编码每一段,并使用注意力或 RNN 来获取长文本表示以进行分类。在本工作中,我们提出了一种简单但有效的模型——段感知多维感知机(SWIPE),以替代上述框架中的注意力/RNN。与以往努力不同,SWIPE 能通过监督训练有效学习整个文本的标签,同时以无监督方式感知各段的标签并估计它们对长文本标注的贡献。作为通用分类器,SWIPE 支持不同的编码器,并在分类准确率和模型效率方面优于 SOTA 模型。值得注意的是,SWIPE 实现了卓越的可解释性,使长文本分类结果透明化。

英文摘要:

Because of the inevitable cost and complexity of transformer and pre-trained models, efficiency concerns are raised for long text classification. Meanwhile, in the highly sensitive domains, e.g., healthcare and legal long-text mining, potential model distrust, yet underrated and underexplored, may hatch vital apprehension. Existing methods generally segment the long text, encode each piece with the pre-trained model, and use attention or RNNs to obtain long text representation for classification. In this work, we propose a simple but effective model, Segment-aWare multIdimensional PErceptron (SWIPE), to replace attention/RNNs in the above framework. Unlike prior efforts, SWIPE can effectively learn the label of the entire text with supervised training, while perceive the labels of the segments and estimate their contributions to the long-text labeling in an unsupervised manner. As a general classifier, SWIPE can endorse different encoders, and it outperforms SOTA models in terms of classification accuracy and model efficiency. It is noteworthy that SWIPE achieves superior interpretability to transparentize long text classification results.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑