A Manually-Curated Dataset of Fixes to Vulnerabilities of Open-Source Software
SAP Security Research（SAP安全研究部门）
Serena E. Ponta, Henrik Plate, Antonino Sabetta, Michele Bezzi, Cédric Dangremont
备注：
This is a pre-print version of the paper that appears in the proceedings of The 16th International Conference on Mining Software Repositories (MSR), Data Showcase track
发表信息：
Proceedings of The 16th International Conference on Mining Software Repositories (Data Showcase track), 2019
增进我们对软件漏洞的理解、实现漏洞识别的自动化、分析其影响并最终缓解漏洞，对于开发更安全的软件是必要的。在运行我们开发的、目前被 SAP 数百个开发部门使用的漏洞评估工具期间，我们人工收集并整理了一个开源软件漏洞及其修复提交的数据集。这些数据既来自 National Vulnerability Database（NVD，国家漏洞数据库），也来自我们持续监控的项目特定 Web 资源。基于这些数据，我们提取出一个数据集，将影响 205 个不同开源 Java 项目的 624 个公开披露漏洞映射到修复它们的 1282 个提交；这些项目用于 SAP 产品或内部工具。在 624 个漏洞中，有 29 个完全没有 CVE 标识符，另有 46 个虽已由编号机构分配 CVE 标识符，但尚未出现在 NVD 中。该数据集以开源许可证发布，并附带支持脚本，使研究人员能够从相应仓库自动获取提交的实际内容，并扩充每个实例可用的属性。此外，这些脚本还允许用并非安全修复的额外实例补充数据集（例如，这在机器学习应用中很有用）。我们的数据集已成功用于训练分类器，这些分类器能够自动识别代码仓库中与安全相关的提交。将此数据集和支持代码开源发布，将使未来研究能够基于具有工业相关性的数据；同时，这也是朝着让开源社区、学术界和工业界共同维护该数据集迈出的具体一步。
英文摘要：
Advancing our understanding of software vulnerabilities, automating their identification, the analysis of their impact, and ultimately their mitigation is necessary to enable the development of software that is more secure. While operating a vulnerability assessment tool that we developed and that is currently used by hundreds of development units at SAP, we manually collected and curated a dataset of vulnerabilities of open-source software and the commits fixing them. The data was obtained both from the National Vulnerability Database (NVD) and from project-specific Web resources that we monitor on a continuous basis. From that data, we extracted a dataset that maps 624 publicly disclosed vulnerabilities affecting 205 distinct open-source Java projects, used in SAP products or internal tools, onto the 1282 commits that fix them. Out of 624 vulnerabilities, 29 do not have a CVE identifier at all and 46, which do have a CVE identifier assigned by a numbering authority, are not available in the NVD yet. The dataset is released under an open-source license, together with supporting scripts that allow researchers to automatically retrieve the actual content of the commits from the corresponding repositories and to augment the attributes available for each instance. Also, these scripts allow to complement the dataset with additional instances that are not security fixes (which is useful, for example, in machine learning applications). Our dataset has been successfully used to train classifiers that could automatically identify security-relevant commits in code repositories. The release of this dataset and the supporting code as open-source will allow future research to be based on data of industrial relevance; also, it represents a concrete step towards making the maintenance of this dataset a shared effort involving open-source communities, academia, and the industry.
MVD: A Multi-Lingual Software Vulnerability Detection Framework
TikTok；CREST - Centre for Research on Engineering Software Technologies（CREST——工程软件技术研究中心）；The University of Adelaide（阿德莱德大学）；Cyber Security Cooperative Research Centre（网络安全合作研究中心）
Data-driven research on the automated discovery and repair of security vulnerabilities in source code requires comprehensive datasets of real-life vulnerable code and their fixes. To assist in such research, we propose a method to automatically collect and curate a comprehensive vulnerability dataset from Common Vulnerabilities and Exposures (CVE) records in the public National Vulnerability Database (NVD). We implement our approach in a fully automated dataset collection tool and share an initial release of the resulting vulnerability dataset named CVEfixes. The CVEfixes collection tool autom…
LiteLMGuard: Seamless and Lightweight On-Device Prompt Filtering for Safeguarding Small Language Models against Quantization-induced Risks and Vulnerabilities
SPIES Research Lab, Dept. of CSE, Texas A&M University（SPIES研究实验室，计算机科学与工程系，德克萨斯大学阿马尔科分校）
Due to the risks associated with vulnerabilities in smart contracts, their security has gained significant attention in recent years. However, there is a lack of open datasets on smart contract vulnerabilities and their fixes that allows for data-driven research. Towards this end, we propose an automated method for mining and classifying Ethereum's smart contract vulnerabilities and their corresponding fixes from GitHub and from the Common Vulnerabilities and Exposures (CVE) records in the National Vulnerability Database. We implemented the proposed method in a fully automated framework, which…
Users around the world rely on software-intensive systems in their day-to-day activities. These systems regularly contain bugs and security vulnerabilities. To facilitate bug fixing, data-driven models of automatic program repair use pairs of buggy and fixed code to learn transformations that fix errors in code. However, automatic repair of security vulnerabilities remains under-explored. In this work, we propose ways to improve code representations for vulnerability repair from three perspectives: input data type, data-driven models, and downstream tasks. The expected results of this work are…
With the increasing disclosure of vulnerabilities in open-source software, software composition analysis (SCA) has been widely applied to reveal third-party libraries and the associated vulnerabilities in software projects. Beyond the revelation, SCA tools adopt various remediation strategies to fix vulnerabilities, the quality of which varies substantially. However, ineffective remediation could induce side effects, such as compilation failures, which impede acceptance by users. According to our studies, existing SCA tools could not correctly handle the concerns of users regarding the compati…
Background: Security regressions are vulnerabilities introduced in a previously unaffected software system. They often happen as a result of source code changes (e.g., a bug fix) and can have severe effects. Aims: To increase the understanding of security regressions. This is an important step in developing secure software engineering. Method: We perform an exploratory, mixed-method case study of Mozilla. First, we analyze 78 regression vulnerabilities and 72 bug reports where a bug fix introduced a regression vulnerability at Mozilla. We investigate how developers interact in these bug report…
Third-party library dependencies are commonplace in today's software development. With the growing threat of security vulnerabilities, applying security fixes in a timely manner is important to protect software systems. As such, the community developed a list of software and hardware weakness known as Common Weakness Enumeration (CWE) to assess vulnerabilities. Prior work has revealed that maintenance activities such as refactoring code potentially correlate with security-related aspects in the source code. In this work, we explore the relationship between refactoring and security by analyzing…
Public development processes are a key characteristic of open source projects. However, fixes for vulnerabilities are usually discussed privately among a small group of trusted maintainers, and integrated without prior public involvement. This is supposed to prevent early disclosure, and cope with embargo and non-disclosure agreement (NDA) rules. While regular development activities leave publicly available traces, fixes for vulnerabilities that bypass the standard process do not. We present a data-mining based approach to detect code fragments that arise from such infringements of the standar…
The increasing interest in open source software has led to the emergence of large language-specific package distributions of reusable software libraries, such as npm and RubyGems. These software packages can be subject to vulnerabilities that may expose dependent packages through explicitly declared dependencies. Using Snyk's vulnerability database, this article empirically studies vulnerabilities affecting npm and RubyGems packages. We analyse how and when these vulnerabilities are disclosed and fixed, and how their prevalence changes over time. We also analyse how vulnerable packages expose …
KTH Royal Institute of Technology（瑞典皇家理工学院）；Colorado State University（科罗拉多州立大学）
In this paper, we address the problem of automatic repair of software vulnerabilities with deep learning. The major problem with data-driven vulnerability repair is that the few existing datasets of known confirmed vulnerabilities consist of only a few thousand examples. However, training a deep learning model often requires hundreds of thousands of examples. In this work, we leverage the intuition that the bug fixing task and the vulnerability fixing task are related and that the knowledge learned from bug fixes can be transferred to fixing vulnerabilities. In the machine learning community, …
Security is a requirement of utmost importance to produce high-quality software. However, there is still a considerable amount of vulnerabilities being discovered and fixed almost weekly. We hypothesize that developers affect the maintainability of their codebases when patching vulnerabilities. This paper evaluates the impact of patches to improve security on the maintainability of open-source software. Maintainability is measured based on the Better Code Hub's model of 10 guidelines on a dataset, including 1300 security-related commits. Results show evidence of a trade-off between security an…
Java platform provides various APIs to facilitate secure coding. However, correctly using security APIs is usually challenging for developers who lack cybersecurity training. Prior work shows that many developers misuse security APIs; such misuses can introduce vulnerabilities into software, void security protections, and present security exploits to hackers. To eliminate such API-related vulnerabilities, this paper presents SEADER -- our new approach that detects and repairs security API misuses. Given an exemplar, insecure code snippet, and its secure counterpart, SEADER compares the snippet…
Tilburg University（蒂尔堡大学）；SAP Security Research（SAP安全研究院）；Eindhoven University of Technology（埃因霍温理工大学）
The sources of reliable, code-level information about vulnerabilities that affect open-source software (OSS) are scarce, which hinders a broad adoption of advanced tools that provide code-level detection and assessment of vulnerable OSS dependencies. In this paper, we study the extent to which the output of off-the-shelf static code analyzers can be used as a source of features to represent commits in Machine Learning (ML) applications. In particular, we investigate how such features can be used to construct embeddings and train ML models to automatically identify source code commits that cont…
This paper presents results from the MSR 2021 Hackathon. Our team investigates files/projects that contain known security vulnerabilities and how widespread they are throughout repositories in open source software. These security vulnerabilities can potentially be propagated through code reuse even when the vulnerability is fixed in different versions of the code. We utilize the World of Code infrastructure to discover file-level duplication of code from a nearly complete collection of open source software. This paper describes a method and set of tools to find all open source projects that us…
The absolute majority of software today is developed collaboratively using collaborative version control tools such as Git. It is a common practice that once a vulnerability is detected and fixed, the developers behind the software issue a Common Vulnerabilities and Exposures or CVE record to alert the user community of the security hazard and urge them to integrate the security patch. However, some companies might not disclose their vulnerabilities and just update their repository. As a result, users are unaware of the vulnerability and may remain exposed. In this paper, we present a system t…
Context: The Heartbleed vulnerability brought OpenSSL to international attention in 2014. The almost moribund project was a key security component in public web servers and over a billion mobile devices. This vulnerability led to new investments in OpenSSL. Objective: The goal of this study is to determine how the Heartbleed vulnerability changed the software evolution of OpenSSL. We study changes in vulnerabilities, code quality, project activity, and software engineering practices. Method: We use a mixed methods approach, collecting multiple types of quantitative data and qualitative data fr…
The exploitation of smart-contract vulnerabilities can have catastrophic consequences such as the loss of millions of pounds worth of crypto assets. Formal verification can be a useful tool in identifying vulnerabilities and proving that they have been fixed. In this paper, we present a formalisation of Solidity and the Ethereum blockchain using the Solid language and its blockchain; a Solid program is obtained by explicating/desugaring a Solidity program. We make some abstractions that over-approximate the way in which Solidity/Ethereum behave. Based on this formalisation, we create Solidifie…
Security patches in open-source software, providing security fixes to identified vulnerabilities, are crucial in protecting against cyberattacks. Despite the National Vulnerability Database (NVD) publishes identified vulnerabilities, a vast majority of vulnerabilities and their corresponding security patches remain beyond public exposure, e.g., in the open-source libraries that are heavily relied on by developers. An extensive security patches dataset could help end-users such as security companies, e.g., building a security knowledge base, or researchers, e.g., aiding in vulnerability researc…
