Senatus --一个快速准确的代码到代码推荐引擎
Senatus -- A Fast and Accurate Code-to-Code Recommendation Engine
摘要:
基于源代码的机器学习(MLOnCode)是一个受欢迎的研究领域,它受到大规模代码存储库的可用性以及用于挖掘源代码的强大概率和深度学习模型的开发的推动。代码到代码推荐是MLOnCode中的一项任务,旨在推荐相关、多样和简洁的代码片段,这些代码片段可以有效地扩展开发人员当前在其开发环境(IDE)中编写的代码。代码到代码推荐引擎通过减少IDE的上下文切换和增加代码重用,有望提高开发人员的生产力。现有的代码到代码推荐引擎不能优雅地扩展到大的代码库,随着代码存储库大小的增加,查询时间呈现线性增长。此外,现有的代码到代码推荐引擎未能在排名功能中考虑代码存储库的全局统计数据,例如代码片段长度的分布,从而导致次优的检索结果。我们使用\emph{Senatus}(一种新的代码到代码推荐引擎)来解决这两个弱点。Senatus的核心是\emph{De-Skew} LSH,这是一种新的局部敏感散列(LSH)算法,它为快速(次线性时间)检索数据编制索引,同时还使用基于抽象语法树的特征评分和选择算法来抵消片段长度分布中的偏斜。我们对Senatus进行了评估,发现推荐的质量高于竞争基线,同时实现了更快的搜索。例如,在CodeSearchNet数据集上,与Facebook Aroma相比,Senatus将性能提高了31.21\% F1和147.9 \emph{x}更快的查询时间。Senatus也比标准MinHash LSH快29.2\% F1和快51.02 \emph{x}的查询时间。
英文摘要:
Machine learning on source code (MLOnCode) is a popular research field that has been driven by the availability of large-scale code repositories and the development of powerful probabilistic and deep learning models for mining source code. Code-to-code recommendation is a task in MLOnCode that aims to recommend relevant, diverse and concise code snippets that usefully extend the code currently being written by a developer in their development environment (IDE). Code-to-code recommendation engines hold the promise of increasing developer productivity by reducing context switching from the IDE and increasing code-reuse. Existing code-to-code recommendation engines do not scale gracefully to large codebases, exhibiting a linear growth in query time as the code repository increases in size. In addition, existing code-to-code recommendation engines fail to account for the global statistics of code repositories in the ranking function, such as the distribution of code snippet lengths, leading to sub-optimal retrieval results. We address both of these weaknesses with \emph{Senatus}, a new code-to-code recommendation engine. At the core of Senatus is \emph{De-Skew} LSH a new locality sensitive hashing (LSH) algorithm that indexes the data for fast (sub-linear time) retrieval while also counteracting the skewness in the snippet length distribution using novel abstract syntax tree-based feature scoring and selection algorithms. We evaluate Senatus and find the recommendations to be of higher quality than competing baselines, while achieving faster search. For example on the CodeSearchNet dataset Senatus improves performance by 31.21\% F1 and 147.9\emph{x} faster query time compared to Facebook Aroma. Senatus also outperforms standard MinHash LSH by 29.2\% F1 and 51.02\emph{x} faster query time.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。