arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2102.09991cs.CL
收藏

基于Transformer的包围学习在科技论文分类中的应用

Using Transformer based Ensemble Learning to classify Scientific Articles

Fidelity Investments(富达投资)

Sohom Ghosh, Ankush Chopra

摘要:

很多时候,审稿人不能欣赏研究人员的新颖想法,也不能提供一般性的反馈。因此,有必要根据审查员的专业领域适当分配审查员。此外,从头到尾阅读每一篇论文并将其分配给审稿人是一项繁琐的任务。在本文中,我们描述了我们的团队FideLIPI在SDPRA-2021的共享任务中提交的系统[14]。它包括四个独立的子系统,能够将科学文献的摘要分类到给定的七个类别之一。第一个是在这些抽象之上构建的基于RoBERTA [10]的模型。在第一个模型中添加基于主题模型/潜在狄利克雷分配(LDA)[2]的功能,从而产生第二个子系统。第三个是句子级的RoberTa [10]模型。第四个是使用词频逆文档频率(TF-IDF)特征构建的Logistic回归模型。我们集成预测这四个子系统使用多数表决,以开发最终的系统,它给出了一个F1的测试和验证集的分数为0.93。在验证集上的F1得分方面,它优于现有的最先进(SOTA)模型SciBERT [1]。https://github.com/SDPRA-2021/shared-task/tree/main/FideLIPI

英文摘要:

Many time reviewers fail to appreciate novel ideas of a researcher and provide generic feedback. Thus, proper assignment of reviewers based on their area of expertise is necessary. Moreover, reading each and every paper from end-to-end for assigning it to a reviewer is a tedious task. In this paper, we describe a system which our team FideLIPI submitted in the shared task of SDPRA-2021 [14]. It comprises four independent sub-systems capable of classifying abstracts of scientific literature to one of the given seven classes. The first one is a RoBERTa [10] based model built over these abstracts. Adding topic models / Latent dirichlet allocation (LDA) [2] based features to the first model results in the second sub-system. The third one is a sentence level RoBERTa [10] model. The fourth one is a Logistic Regression model built using Term Frequency Inverse Document Frequency (TF-IDF) features. We ensemble predictions of these four sub-systems using majority voting to develop the final system which gives a F1 score of 0.93 on the test and validation set. This outperforms the existing State Of The Art (SOTA) model SciBERT's [1] in terms of F1 score on the validation set.Our codebase is available at https://github.com/SDPRA-2021/shared-task/tree/main/FideLIPI

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑