This research analyzes complex networks in open-source software at the inter-package level, where package dependencies often span across projects and between development groups. We review complex networks identified at ``lower'' levels of abstraction, and then formulate a description of interacting software components at the package level, a relatively ``high'' level of abstraction. By mining open-source software repositories from two sources, we empirically show that the coupling of modules at this granularity creates a small-world and scale-free network in both instances.
The R package ecosystem is expanding fast and dependencies among packages in the ecosystem are becoming more complex. In this study, we explored the package dependencies from a new aspect. We applied a new metric named "dependency heaviness" which measures the number of additional strong dependencies that a package uniquely contributes to its child or downstream packages. It also measures the total reduced dependencies in the ecosystem when the role of a package is changed from a strong parent to a weak parent. We systematically studied how the dependency heaviness spreads from parent to child…
An Empirical Comparison of Dependency Network Evolution in Seven Software Packaging Ecosystems
几乎每一种流行的程序设计语言都带有一个或多个包管理器。由这样的包管理器分发的软件包形成大的软件生态系统。这些打包生态系统包含大量定期更新的包版本，并且与其他包版本有许多依赖关系。虽然打包生态系统对各自的开发人员社区非常有用，但它们面临着与规模、复杂性和进化速度相关的挑战。典型的问题是向后不兼容的软件包更新，以及依赖于已经过时或不活动的软件包的风险。本文使用libraries.io数据集对七个不同规模和年龄的包装生态系统的包依赖网络演变之间的相似性和差异性进行了定量实证分析：Cargo for Rust，CPAN for Perl，CRAN for R，npm for JavaScript，NuGet for.NET平台，Packagist for PHP和RubyGems for Ruby。我们提出了新的指标来捕捉这些依赖网络的增长，易变性，可恢复性和脆弱性，并使用这些指标来分析和比较它们的演变。我们观察到，依赖网络往往会随着时间的推移而增长，无论是在大小和软件包更新的数量，而少数软件包负责大部分的软件包更新。大多数软件包依赖于其他软件包，但只有一小部分软件包占大多数反向依赖。我们观察到一个高比例的脆弱的软件包，由于高和越来越多的传递依赖。这些发现有助于评估软件包依赖网络的质量，并通过依赖管理工具和强加的政策来改善它。
Modern programming languages such as Java, JavaScript, and Rust encourage software reuse by hosting diverse and fast-growing repositories of highly interdependent packages (i.e., reusable libraries) for their users. The standard way to study the interdependence between software packages is to infer a package dependency network by parsing manifest data. Such networks help answer questions such as "How many packages have dependencies to packages with known security issues?" or "What are the most used packages?". However, an overlooked aspect in existing studies is that manifest-inferred relation…
Dependency solving is a hard (NP-complete) problem in all non-trivial component models due to either mutually incompatible versions of the same packages or explicitly declared package conflicts. As such, software upgrade planning needs to rely on highly specialized dependency solvers, lest falling into pitfalls such as incompleteness-a combination of package versions that satisfy dependency constraints does exist, but the package manager is unable to find it. In this paper we look back at proposals from dependency solving research dating back a few years. Specifically, we review the idea of tr…
The increasing interest in open source software has led to the emergence of large language-specific package distributions of reusable software libraries, such as npm and RubyGems. These software packages can be subject to vulnerabilities that may expose dependent packages through explicitly declared dependencies. Using Snyk's vulnerability database, this article empirically studies vulnerabilities affecting npm and RubyGems packages. We analyse how and when these vulnerabilities are disclosed and fixed, and how their prevalence changes over time. We also analyse how vulnerable packages expose …
Software packages developed and distributed through package managers extensively depend on other packages. These dependencies are regularly updated, for example to add new features, resolve bugs or fix security issues. In order to take full advantage of the benefits of this type of reuse, developers should keep their dependencies up to date by relying on the latest releases. In practice, however, this is not always possible, and packages lag behind with respect to the latest version of their dependencies. This phenomenon is described as technical lag in the literature. In this paper, we perfor…
Modern scientific software stacks have become extremely complex, using many programming models and libraries to exploit a growing variety of GPUs and accelerators. Package managers can mitigate this complexity using dependency solvers, but they are reaching their limits. Finding compatible dependency versions is NP-complete, and modeling the semantics of package compatibility modulo build-time options, GPU runtimes, flags, and other parameters is extremely difficult. Within this enormous configuration space, defining a "good" configuration is daunting. We tackle this problem using Answer Set P…
Third party libraries are used to integrate existing solutions for common problems and help speed up development. The use of third party libraries, however, can carry risks, for example through vulnerabilities in these libraries. Studying the dependency networks of package managers lets us better understand and mitigate these risks. So far, the dependency networks of the three most important package managers of the Apple ecosystem, CocoaPods, Carthage and Swift PM, have not been studied. We analysed the dependencies for all publicly available open source libraries up to December 2021 and compi…
As the complexity and size of software projects increases in real-world environments, maintaining and creating maintainable and dependable code becomes harder and more costly. Refactoring is considered as a method for enhancing the internal structure of code for improving many software properties such as maintainability. In this thesis, the subject of refactoring software packages using community detection algorithms is discussed, with a focus on the notion of package stability. The proposed algorithm starts by extracting a package dependency network from Java byte code and a community detecti…
Third-party libraries (TPLs) are frequently reused in software to reduce development cost and the time to market. However, external library dependencies may introduce vulnerabilities into host applications. The issue of library dependency has received considerable critical attention. Many package managers, such as Maven, Pip, and NPM, are proposed to manage TPLs. Moreover, a significant amount of effort has been put into studying dependencies in language ecosystems like Java, Python, and JavaScript except C/C++. Due to the lack of a unified package manager for C/C++, existing research has only…
As modern software extensively uses free open source packages as dependencies, developers have to regularly pull in new third-party code through frequent updates. However, without a proper review of every incoming change, vulnerable and malicious code can sneak into the codebase through these dependencies. The goal of this study is to aid developers in securely accepting dependency updates by measuring if the code changes in an update have passed through a code review process. We implement Depdive, an update audit tool for packages in Crates.io, npm, PyPI, and RubyGems registry. Depdive first …
With the increase of complexity of modern software, social collaborative coding and reuse of open source software packages become more and more popular, which thus greatly enhances the development efficiency and software quality. However, the explosive growth of open source software packages exposes developers to the challenge of information overload. While this can be addressed by conventional recommender systems, they usually do not consider particular constraints of social coding such as social influence among developers and dependency relations among software packages. In this paper, we ai…
University of Southern California（南加州大学）；University of California, Berkeley（加州大学伯克利分校）
A network embedding consists of a vector representation for each node in the network. Its usefulness has been shown in many real-world application domains, such as social networks and web networks. Directed networks with text associated with each node, such as software package dependency networks, are commonplace. However, to the best of our knowledge, their embeddings have hitherto not been specifically studied. In this paper, we propose PCTADW-1 and PCTADW-2, two algorithms based on neural networks that learn embeddings of directed networks with text associated with each node. We create two …
Contemporary software is characterized by reuse of components that are declared as dependencies and that are received from package managers/registries, such as, NPM, PyPI, RubyGems, Maven Central, etc. Direct and indirect dependency relations often form opaque dependency networks, that sometimes lead to conflicting software licenses within these. In this paper, we study license use and license incompatibilities between all components from seven package registries (Cargo, Maven, NPM, NuGet, Packagist, PyPI, RubyGems) with a closer investigation of license incompatibilities caused by the GNU Aff…
ROOT is a large code base with a complex set of build-time dependencies; there is a significant difference in compilation time between the "core" of ROOT and the full-fledged deployment. We present results on a "delayed build" for internal ROOT packages and external packages. This gives the ability to offer a "lightweight" core of ROOT, later extended by building additional modules to extend the functionality of ROOT. As a part of this work, we have improved the separation of ROOT code into distinct modules and packages with minimal dependencies. This approach gives users better flexibility an…
Popular adoption of third-party libraries for contemporary software development has led to the creation of large inter-dependency networks, where sustainability issues of a single library can have widespread network effects. Maintainers of these libraries are often overworked, relying on the contributions of volunteers to sustain these libraries. In this work, we measure contributions that are aligned with dependency changes, to understand where they come from (i.e., non-maintainer, client maintainer, library maintainer, and library and client maintainer), analyze whether they contribute to li…
Reusing software libraries is a pillar of modern software engineering. In 2022, the average Java application depends on 40 third-party libraries. Relying on such libraries exposes a project to potential vulnerabilities and may put an application and its users at risk. Unfortunately, research on software ecosystems has shown that the number of projects that are affected by such vulnerabilities is rising. Previous investigations usually reason about dependencies on the dependency level, but we believe that this highly inflates the actual number of affected projects. In this work, we study the ef…
Cargo, the software packaging manager of Rust, provides a yank mechanism to support release-level deprecation, which can prevent packages from depending on yanked releases. Most prior studies focused on code-level (i.e., deprecated APIs) and package-level deprecation (i.e., deprecated packages). However, few studies have focused on release-level deprecation. In this study, we investigate how often and how the yank mechanism is used, the rationales behind its usage, and the adoption of yanked releases in the Cargo ecosystem. Our study shows that 9.6% of the packages in Cargo have at least one y…
Modern software systems are often built by leveraging code written by others in the form of libraries and packages to accelerate their development. While there are many benefits to using third-party packages, software projects often become dependent on a large number of software packages. Consequently, developers are faced with the difficult challenge of maintaining their project dependencies by keeping them up-to-date and free of security vulnerabilities. However, how often are project dependencies used in production where they could pose a threat to their project's security? We conduct an em…
EvidenT: An Evidence-Preserving Framework for Iterative System-Level Package Repair
Nankai University（南开大学）；Microsoft（微软）；Haihe Laboratory of Information Technology Application Innovation（海河信息技术应用创新实验室）；Key Laboratory of Data and Intelligent System Security, Ministry of Education（教育部数据与智能系统安全重点实验室）；Tianjin Key Laboratory of Software Experience and Human Computer Interaction（天津市软件体验与人机交互重点实验室）；Tsinghua University（清华大学）
