Source code clones pose risks ranging from intellectual property violations to unintended vulnerabilities. Effective and efficient scalable clone detection, especially for diverged clones, remains challenging. Large language models (LLMs) have recently been applied to clone detection tasks. However, the rapid emergence of LLMs raises questions about optimal model selection and potential LLM-ensemble efficacy.
This paper addresses the first question by identifying 76 LLMs and filtering them down to suitable candidates for large-scale clone detection. The candidates were evaluated on two public industrial datasets, BigCloneBench, and a commercial large-scale dataset. No uniformly 'best-LLM' emerged, though CodeT5+110M, CuBERT and SPTCode were top-performers. Analysis of LLM-candidates suggested that smaller embedding sizes, smaller tokenizer vocabularies and tailored datasets are advantageous. On commercial large-scale dataset a top-performing CodeT5+110M achieved 39.71\% precision: twice the precision of previously used CodeBERT.
To address the second question, this paper explores ensembling of the selected LLMs: effort-effective approach to improving effectiveness. Results suggest the importance of score normalization and favoring ensembling methods like maximum or sum over averaging. Also, findings indicate that ensembling approach can be statistically significant and effective on larger datasets: the best-performing ensemble achieved even higher precision of 46.91\% over individual LLM on the commercial large-scale code.
成功的跨语言克隆检测可以使研究人员和开发人员创建强大的语言迁移工具,一旦掌握了一种程序设计语言,就可以方便地学习其他语言,并促进代码片段在更广泛的代码库中的重用。然而,识别跨语言克隆提出了特殊的挑战,克隆检测问题。任意语言之间缺乏公共的底层表示意味着检测克隆需要以下解决方案之一:1)跨每个目标语言复制的静态分析框架,其注释匹配所有语言的语言特征,或者2)基于运行时行为检测克隆的动态分析框架。在这项工作中,我们证明了后者的解决方案,一个动态分析方法称为SLACC跨语言克隆检测的可行性。与之前的克隆检测技术一样,我们使用输入/输出行为来匹配克隆,尽管我们通过放大输入的数量和覆盖更多的数据类型来克服之前工作的局限性;因此,实现了比之前尝试更好的聚类。由于聚类是基于输入/输出行为生成的,因此SLACC支持跨语言克隆检测。作为一个额外的挑战,我们的目标是静态类型语言Java和动态类型语言Python。与最近的Java克隆检测工具HitoshiIO相比,SLACC检索的集群数量是HitoshiIO的6倍,并且精度更高(86.7% vs. 30.7%)。这是第一个对动态类型语言执行克隆检测的工作(精度= 87.3%),也是第一个对缺乏公共底层表示的语言执行克隆检测的工作(精度= 94.1%)。它为实现可伸缩语言迁移工具的更大目标迈出了第一步。
MuRating: A High Quality Data Selecting Approach to Multilingual Large Language Model Pretraining
Hong Kong University of Science and Technology (Guangzhou)(香港理工大学(广州));ByteDance(字节跳动);Eindhoven University of Technology(埃因霍温理工大学);University of Melbourne(墨尔本大学);University of Liverpool(利物浦大学)