公共Source Code随时间的增长和复制：大规模来源追踪
Growth and Duplication of Public Source Code over Time: Provenance Tracking at Scale
摘要：
我们研究了最大的已知公开源代码语料库的演变，即，软件遗产档案（4B唯一的源代码文件，1B提交捕获了50M软件项目的开发历史）。在这样的语料库中，我们量化了原始的、从未见过的源代码文件和提交的增长率。我们发现，在40多年的时间里，增长率呈指数增长。然后，我们估计了倍增因子，即，多少相同的伪像（例如，文件或提交）出现在不同的上下文中（例如，提交或源代码分发位置）。我们观察到在不同提交中相同源代码文件的倍增中的组合爆炸。我们讨论了这些发现对跟踪源代码工件的起源问题的影响（例如，在何处以及何时给定的源代码文件或提交已经在野外被观察到）。为此，我们对不同的数据模型进行基准测试，以捕获这种规模和增长率下的软件起源信息。我们确定了一个可行的解决方案，是部署在商品硬件，似乎是可预见的未来维护。
英文摘要：
We study the evolution of the largest known corpus of publicly available source code, i.e., the Software Heritage archive (4B unique source code files, 1B commits capturing their development histories across 50M software projects). On such corpus we quantify the growth rate of original, never-seen-before source code files and commits. We find the growth rates to be exponential over a period of more than 40 years.We then estimate the multiplication factor, i.e., how much the same artifacts (e.g., files or commits) appear in different contexts (e.g., commits or source code distribution places). We observe a combinatorial explosion in the multiplication of identical source code files across different commits.We discuss the implication of these findings for the problem of tracking the provenance of source code artifacts (e.g., where and when a given source code file or commit has been observed in the wild) for the entire body of publicly available source code. To that end we benchmark different data models for capturing software provenance information at this scale and growth rate. We identify a viable solution that is deployable on commodity hardware and appears to be maintainable for the foreseeable future.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
