The widespread adoption of open-source software (OSS) has accelerated software innovation but also increased security risks due to the rapid propagation of vulnerabilities and silent patch releases. In recent years, large language models (LLMs) and LLM-based agents have demonstrated remarkable capabilities in various software engineering (SE) tasks, enabling them to effectively address software security challenges such as vulnerability detection. However, systematic evaluation of the capabilities of LLMs and LLM-based agents in security patch detection remains limited. To bridge this gap, we conduct a comprehensive evaluation of the performance of LLMs and LLM-based agents for security patch detection. Specifically, we investigate three methods: Plain LLM (a single LLM with a system prompt), Data-Aug LLM (data augmentation based on the Plain LLM), and the ReAct Agent (leveraging the thought-action-observation mechanism). We also evaluate the performance of both commercial and open-source LLMs under these methods and compare these results with those of existing baselines. Furthermore, we analyze the detection performance of these methods across various vulnerability types, and examine the impact of different prompting strategies and context window sizes on the results. Our findings reveal that the Data-Aug LLM achieves the best overall performance, whereas the ReAct Agent demonstrates the lowest false positive rate (FPR). Although baseline methods exhibit strong accuracy, their false positive rates are significantly higher. In contrast, our evaluated methods achieve comparable accuracy while substantially reducing the FPR. These findings provide valuable insights into the practical applications of LLMs and LLM-based agents in security patch detection, highlighting their advantage in maintaining robust performance while minimizing false positive rates.
With the increasing usage of open-source software (OSS) components, vulnerabilities embedded within them are propagated to a huge number of underlying applications. In practice, the timely application of security patches in downstream software is challenging. The main reason is that such patches do not explicitly indicate their security impacts in the documentation, which would be difficult to recognize for software maintainers and users. However, attackers can still identify these "secret" security patches by analyzing the source code and generate corresponding exploits to compromise not only…
Training machine learning approaches for vulnerability identification and producing reliable tools to assist developers in implementing quality software -- free of vulnerabilities -- is challenging due to the lack of large datasets and real data. Researchers have been looking at these issues and building datasets. However, these datasets usually miss natural language artifacts and programming language diversity. We scraped the entire CVE details database for GitHub references and augmented the data with 3 security-related datasets. We used the data to create a ground-truth dataset of natural l…
With the rapid development of the Internet of Things, there are many interacting devices and applications. One crucial challenge is how to provide security. Our proposal for a new direction is to create "smart buttons" and collections of them called "smart blankets" as hardware/software security patches rather than software-only patches.
The absolute majority of software today is developed collaboratively using collaborative version control tools such as Git. It is a common practice that once a vulnerability is detected and fixed, the developers behind the software issue a Common Vulnerabilities and Exposures or CVE record to alert the user community of the security hazard and urge them to integrate the security patch. However, some companies might not disclose their vulnerabilities and just update their repository. As a result, users are unaware of the vulnerability and may remain exposed. In this paper, we present a system t…
In this paper, we perform a comprehensive study of 2,470 patched Android vulnerabilities that we collect from different data sources such as Android security bulletins, CVEDetails, Qualcomm Code Aurora, AOSP Git repository, and Linux Patchwork. In our data analysis, we focus on determining the affected layers, OS versions, severity levels, and common weakness enumerations (CWE) associated with the patched vulnerabilities. Further, we assess the timeline of each vulnerability, including discovery and patch dates. We find that (i) even though the number of patched vulnerabilities changes conside…
Automated Detection of Configuration-Specific Security Vulnerabilities via Patch Analysis
Federal University of Bahia（巴伊亚联邦大学）；University of Notre Dame（圣母大学）；Federal Rural University of Pernambuco（伯南布哥联邦农村大学）；Federal University of Rio de Janeiro（里约热内卢联邦大学）
Security is a requirement of utmost importance to produce high-quality software. However, there is still a considerable amount of vulnerabilities being discovered and fixed almost weekly. We hypothesize that developers affect the maintainability of their codebases when patching vulnerabilities. This paper evaluates the impact of patches to improve security on the maintainability of open-source software. Maintainability is measured based on the Better Code Hub's model of 10 guidelines on a dataset, including 1300 security-related commits. Results show evidence of a trade-off between security an…
Empirical Study of Code Large Language Models for Binary Security Patch Detection
National Key Laboratory for Novel Software Technology at Nanjing University（南京大学新型软件技术国家重点实验室）；Harbin Institute of Technology（哈尔滨工业大学）；Chinese University of Hong Kong（香港中文大学）；Hong Kong University of Science and Technology（香港科学大学）
Data security is one of the most crucial and a major challenge in the digital world. Security, privacy and integrity of data are demanded in every operation performed on internet. Whenever security of data is discussed, it is mostly in the context of secure transfer of data over the unreliable communication networks. But the security of the data in databases is also as important. In this paper we will be presenting some of the common security techniques for the data that can be implemented in fortifying and strengthening the databases.
Cloud computing comes with a lot of advanced features along with privacy and security problem. Smart meter data takes the benefit of cloud computing in the smart grid. User's privacy can be compromised by analyzing the smart meter data generated by household electrical appliances. The user loses control over the data while data is shifted to the cloud. This paper describes the issues under the privacy and security of smart meter data in the cloud environment. We also compare the existing approaches for preserving the privacy and security of smart meter data.
Many open-source projects land security fixes in public repositories before shipping these patches to users. This paper presents attacks on such projects - taking Firefox as a case-study - that exploit patch metadata to efficiently search for security patches prior to shipping. Using access-restricted bug reports linked from patch descriptions, security patches can be immediately identified for 260 out of 300 days of Firefox 3 development. In response to Mozilla obfuscating descriptions, we show that machine learning can exploit metadata such as patch author to search for security patches, ext…
Information Security has become an important issue in modern world as the popularity and infiltration of internet commerce and communication technologies has emerged, making them a prospective medium to the security threats. To surmount these security threats modern data communications uses cryptography an effective, efficient and essential component for secure transmission of information by implementing security parameter counting Confidentiality, Authentication, accountability, and accuracy. To achieve data security different cryptographic algorithms (Symmetric & Asymmetric) are used that ju…
In this paper, we introduce a mechanism that aims to speed up the development cycle of security protocols, by adding automated aid for diagnosis and repair. Our mechanism relies on existing verification tools analyzing intermediate protocols and synthesizing potential attacks if the protocol is flawed. The analysis of these attacks (including type flaw attacks) pinpoints the source of the failure and controls the synthesis of appropriate patches to the protocol. Using strand spaces, we have developed general guidelines for protocol repair, and captured them into formal requirements on (sets of…
We analyze the utilization of publish-subscribe protocols in IoT and Fog Computing and challenges around security configuration, performance, and qualitative characteristics. Such problems with security configuration lead to significant disruptions and high operation costs. Yet, These issues can be prevented by selecting the appropriate transmission technology for each configuration, considering the variations in sizing, installation, sensor profile, distribution, security, networking, and locality. This work aims to present a comparative qualitative and quantitative analysis around diverse co…
With the rapid increasing number of open source software (OSS), the majority of the software vulnerabilities in the open source components are fixed silently, which leads to the deployed software that integrated them being unable to get a timely update. Hence, it is critical to design a security patch identification system to ensure the security of the utilized software. However, most of the existing works for security patch identification just consider the changed code and the commit message of a commit as a flat sequence of tokens with simple neural networks to learn its semantics, while the…
Technology is an endless evolving expression in modern era, which increased security concerns and pushed us to create cyber environment. A National Cyber Security Strategy (NCSS) of a country reflects the state of that country's cyber strength which represents the aim and vision of the cyber security of a country. Formerly, researchers have worked on NCSS by comparing NCSS between different nations for international collaboration and harmonization and some researchers worked on policy framework for their respective governments. However very insignificant attempts had been made to assess the st…
Software defined networking implements the network control plane in an external entity, rather than in each individual device as in conventional networks. This architectural difference implies a different design for control functions necessary for essential network properties, e.g., loop prevention and link redundancy. We explore how such differences redefine the security weaknesses in the SDN control plane and provide a framework for comparative analysis which focuses on essential network properties required by typical production networks. This enables analysis of how these properties are del…
Fuzzing has become one of the most effective bug finding approach for software. In recent years, 24*7 continuous fuzzing platforms have emerged to test critical pieces of software, e.g., Linux kernel. Though capable of discovering many bugs and providing reproducers (e.g., proof-of-concepts), a major problem is that they neglect a critical function that should have been built-in, i.e., evaluation of a bug's security impact. It is well-known that the lack of understanding of security impact can lead to delayed bug fixes as well as patch propagation. In this paper, we develop SyzScope, a system …
Many applications have security vulnerabilities that can be exploited. It is practically impossible to find all of them due to the NP-complete nature of the testing problem. Security solutions provide defenses against these attacks through continuous application testing, fast-patching of vulnerabilities, automatic deployment of patches, and virtual patching detection techniques deployed in network and endpoint security tools. These techniques are limited by the need to find vulnerabilities before the black-hats. We propose an innovative technique to virtually patch vulnerabilities before they …
The main stretch in the paper is buffer overflow anomaly occurring in major source codes, designed in various programming language. It describes the various as to how to improve your code and increase its strength to withstand security theft occurring at vulnerable areas in the code. The main language used is JAVA, regarded as one of the most object oriented language still create lot of error like stack overflow, illegal/inappropriate method overriding. I used tools confined to JAVA to test as how weak points in the code can be rectified before compiled. The byte code theft is difficult to be …
