使用Transformer进行代码质量评估
Code quality assessment using transformers
AI总结:
本文研究利用CodeBERT及任务自适应预训练Transformer模型自动评估Java代码质量,并在新数据集上验证其有效性,证明代码质量具有一定可预测性。
摘要:
使用单元测试和集成测试自动评估编程作业的正确性相当直接。然而,编程任务可以通过多种方式解决,其中许多方式虽然正确,却不够优雅。例如,过多的分支、糟糕的命名或重复性使代码难以理解和维护。这些代码的主观质量很难用当前技术自动评估。在这项工作中,我们研究了使用CodeBERT自动为Java代码分配质量分数。我们尝试了不同的模型和训练范式。我们探索了这些模型在一个用于代码质量评估的新数据集上的准确性。最后,我们使用显著性图评估预测的质量。我们发现代码质量在一定程度上是可预测的,并且使用任务自适应预训练的基于Transformer的模型可以比其他技术更有效地解决该任务。
英文摘要:
Automatically evaluate the correctness of programming assignments is rather straightforward using unit and integration tests. However, programming tasks can be solved in multiple ways, many of which, although correct, are inelegant. For instance, excessive branching, poor naming or repetitiveness make the code hard to understand and maintain. These subjective qualities of code are hard to automatically assess using current techniques. In this work we investigate the use of CodeBERT to automatically assign quality score to Java code. We experiment with different models and training paradigms. We explore the accuracy of the models on a novel dataset for code quality assessment. Finally, we assess the quality of the predictions using saliency maps. We find that code quality to some extent is predictable and that transformer based models using task adapted pre-training can solve the task more efficiently than other techniques.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。