基于April的API评论分类：预训练的Transformer模型能走多远？
Aspect-Based API Review Classification: How Far Can Pre-Trained Transformer Model Go?
摘要：
API（应用程序编程接口）是可重用的软件库，是现代快速软件开发的构建块。以前的研究表明，程序员经常在主流软件问答（Q&A）平台（如Stack Overflow）上分享和搜索API评论，这促使研究人员设计与自动处理API评论相关的任务和方法。在这些任务中，将API审查分类为不同的方面（例如，性能或安全），这被称为基于方面的API审查分类，非常重要。目前最先进的（SOTA）解决方案是基于传统的机器学习算法。受预训练模型在许多软件工程任务中取得的巨大成功的启发，本文针对基于方面的API评审分类任务，对六个预训练模型进行了微调，并在Uddin等人收集的API评审基准上将它们与当前的SOTA解决方案进行了比较（BERT、RoBERTa、ALBERT和XLNet）是在自然语言上进行预训练的，BERTOverflow是在从Stack Overflow上的帖子中提取的文本语料库上进行预训练的，CosSensBERT是为处理不平衡数据而设计的。结果表明，所有六个微调模型优于传统的机器学习为基础的工具。更具体地说，F1评分的改善范围为21.0%至30.2%。我们还发现，BERTOverflow，一个在Stack Overflow的语料库上预先训练的模型，并没有表现出比BERT更好的性能。结果还表明，CosSensBERT在F1方面也没有表现出比BERT更好的性能，但它仍然值得考虑，因为它在MCC和AUC上实现了更好的性能。
英文摘要：
APIs (Application Programming Interfaces) are reusable software libraries and are building blocks for modern rapid software development. Previous research shows that programmers frequently share and search for reviews of APIs on the mainstream software question and answer (Q&A) platforms like Stack Overflow, which motivates researchers to design tasks and approaches related to process API reviews automatically. Among these tasks, classifying API reviews into different aspects (e.g., performance or security), which is called the aspect-based API review classification, is of great importance. The current state-of-the-art (SOTA) solution to this task is based on the traditional machine learning algorithm. Inspired by the great success achieved by pre-trained models on many software engineering tasks, this study fine-tunes six pre-trained models for the aspect-based API review classification task and compares them with the current SOTA solution on an API review benchmark collected by Uddin et al. The investigated models include four models (BERT, RoBERTa, ALBERT and XLNet) that are pre-trained on natural languages, BERTOverflow that is pre-trained on text corpus extracted from posts on Stack Overflow, and CosSensBERT that is designed for handling imbalanced data. The results show that all the six fine-tuned models outperform the traditional machine learning-based tool. More specifically, the improvement on the F1-score ranges from 21.0% to 30.2%. We also find that BERTOverflow, a model pre-trained on the corpus from Stack Overflow, does not show better performance than BERT. The result also suggests that CosSensBERT also does not exhibit better performance than BERT in terms of F1, but it is still worthy of being considered as it achieves better performance on MCC and AUC.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
