arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2007.02599cs.SE
收藏

Sosed:一个查找相似软件项目的工具

Sosed: a tool for finding similar software projects

Egor Bogomolov, Yaroslav Golubev, Artyom Lobanov, Vladimir Kovalenko, Timofey Bryksin

摘要:

在本文中,我们提出了Sosed,用于发现类似的软件项目的工具。我们使用fastText来计算子令牌嵌入到200种语言的120,000个GitHub存储库的密集空间中。然后,我们对嵌入进行聚类,以识别反映源代码中主题的语义相似的子标记组。我们使用900万个GitHub项目的数据集作为参考搜索基础。为了识别类似的项目,我们比较了它们的子令牌之间的集群分布。该工具接收任意项目作为输入,以16种最流行的程序设计语言提取子标记,计算集群分布,并在搜索库中找到具有最接近分布的项目。我们用简短的描述标记了子令牌集群,以使Sosed能够产生可解释的输出。Sosed可在https://github.com/JetBrains-Research/sosed/上获得。该工具演示可在https://www.youtube.com/watch?上获得v=LYLkztCGRt8。子令牌的多语言提取器可在https://github.com/JetBrains-Research/buckwheat/单独获得。

英文摘要:

In this paper, we present Sosed, a tool for discovering similar software projects. We use fastText to compute the embeddings of subtokens into a dense space for 120,000 GitHub repositories in 200 languages. Then, we cluster embeddings to identify groups of semantically similar sub-tokens that reflect topics in source code. We use a dataset of 9 million GitHub projects as a reference search base. To identify similar projects, we compare the distributions of clusters among their sub-tokens. The tool receives an arbitrary project as input, extracts sub-tokens in 16 most popular programming languages, computes cluster distribution, and finds projects with the closest distribution in the search base. We labeled subtoken clusters with short descriptions to enable Sosed to produce interpretable output. Sosed is available at https://github.com/JetBrains-Research/sosed/. The tool demo is available at https://www.youtube.com/watch?v=LYLkztCGRt8. The multi-language extractor of sub-tokens is available separately at https://github.com/JetBrains-Research/buckwheat/.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑