Recent advances in software vulnerability detection have been driven by Language Model (LM)-based approaches. However, these models remain vulnerable to adversarial attacks that exploit lexical and syntax perturbations, allowing critical flaws to evade detection. Existing black-box attacks on LM-based vulnerability detectors primarily rely on isolated perturbation strategies, limiting their ability to efficiently explore the adversarial code space for optimal perturbations. To bridge this gap, we propose HogVul, a black-box adversarial code generation framework that integrates both lexical and syntax perturbations under a unified dual-channel optimization strategy driven by Particle Swarm Optimization (PSO). By systematically coordinating two-level perturbations, HogVul effectively expands the search space for adversarial examples, enhancing the attack efficacy. Extensive experiments on four benchmark datasets demonstrate that HogVul achieves an average attack success rate improvement of 26.05\% over state-of-the-art baseline methods. These findings highlight the potential of hybrid optimization strategies in exposing model vulnerabilities.
This work examines the vulnerability of multimodal (image + text) models to adversarial threats similar to those discussed in previous literature on unimodal (image- or text-only) models. We introduce realistic assumptions of partial model knowledge and access, and discuss how these assumptions differ from the standard "black-box"/"white-box" dichotomy common in current literature on adversarial attacks. Working under various levels of these "gray-box" assumptions, we develop new attack methodologies unique to multimodal classification and evaluate them on the Hateful Memes Challenge classific…
The Pennsylvania State University（宾夕法尼亚州立大学）；Shanghai Jiao Tong University（上海交通大学）；Amplitude（Amplitude公司）
Deep neural networks (DNNs) have been broadly adopted in health risk prediction to provide healthcare diagnoses and treatments. To evaluate their robustness, existing research conducts adversarial attacks in the white/gray-box setting where model parameters are accessible. However, a more realistic black-box adversarial attack is ignored even though most real-world models are trained with private data and released as black-box services on the cloud. To fill this gap, we propose the first black-box adversarial attack method against health risk prediction models named MedAttacker to investigate …
Institute of Computing Technology, CAS（中国科学院计算技术研究所）；Shandong University（山东大学）；RIKEN（理化学研究所）
The binary code similarity detection (BCSD) method measures the similarity of two binary executable codes. Recently, the learning-based BCSD methods have achieved great success, outperforming traditional BCSD in detection accuracy and efficiency. However, the existing studies are rather sparse on the adversarial vulnerability of the learning-based BCSD methods, which cause hazards in security-related applications. To evaluate the adversarial robustness, this paper designs an efficient and black-box adversarial code generation algorithm, namely, FuncFooler. FuncFooler constrains the adversarial…
Recent works have revealed the vulnerability of automatic speech recognition (ASR) models to adversarial examples (AEs), i.e., small perturbations that cause an error in the transcription of the audio signal. Studying audio adversarial attacks is therefore the first step towards robust ASR. Despite the significant progress made in attacking audio examples, the black-box attack remains challenging because only the hard-label information of transcriptions is provided. Due to this limited information, existing black-box methods often require an excessive number of queries to attack a single audio…
MF-CLIP: Leveraging CLIP as Surrogate Models for No-box Adversarial Attacks
Hong Kong University of Science and Technology（香港科技大学）；University of Naples Federico II（那不勒斯费德里科二世大学）；Singapore Management University（新加坡管理大学）；Beijing Jiaotong University（北京交通大学）；National Lab of Pattern Recognition（模式识别国家重点实验室）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）
Translate your gibberish: black-box adversarial attack on machine translation systems
Skolkovo Institute of Science and Technology（斯科尔科沃科学技术研究院）；Institute of Numerical Mathematics, Russian Academy of Sciences（俄罗斯科学院数值数学研究所）；Moscow Institute of Physics and Technology（莫斯科物理技术学院）；Sber AI Lab（俄联邦储蓄银行人工智能实验室）；AIRI
University of Leeds（利兹大学）；Southwest Jiaotong University（西南交通大学）；Zhejiang University（浙江大学）；University of Bath（巴斯大学）
Skeletal motion plays a vital role in human activity recognition as either an independent data source or a complement. The robustness of skeleton-based activity recognizers has been questioned recently, which shows that they are vulnerable to adversarial attacks when the full-knowledge of the recognizer is accessible to the attacker. However, this white-box requirement is overly restrictive in most scenarios and the attack is not truly threatening. In this paper, we show that such threats do exist under black-box settings too. To this end, we propose the first black-box adversarial attack meth…
Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation
PRISM AI（PRISM人工智能公司）；Harmony Intelligence（和谐智能公司）；Leap Laboratories（leap实验室）；MIT CSAIL（麻省理工学院计算机科学与人工智能实验室）；ETH AI Center, ETH Zurich（苏黎世联邦理工学院人工智能中心）
This work explores the signal awareness of AI models for source code understanding. Using a software vulnerability detection use case, we evaluate the models' ability to capture the correct vulnerability signals to produce their predictions. Our prediction-preserving input minimization (P2IM) approach systematically reduces the original source code to a minimal snippet which a model needs to maintain its prediction. The model's reliance on incorrect signals is then uncovered when the vulnerability in the original code is missing in the minimal snippet, both of which the model however predicts …
Zhejiang University of Science and Technology（浙江科技大学）；Zhejiang University（浙江大学）；Guangzhou University（广州大学）；Hangzhou Hikvision Digital Technology Co., Ltd.（杭州海康威视数字技术股份有限公司）
With the boom of edge intelligence, its vulnerability to adversarial attacks becomes an urgent problem. The so-called adversarial example can fool a deep learning model on the edge node to misclassify. Due to the property of transferability, the adversary can easily make a black-box attack using a local substitute model. Nevertheless, the limitation of resource of edge nodes cannot afford a complicated defense mechanism as doing on the cloud data center. To overcome the challenge, we propose a dynamic defense mechanism, namely EI-MTD. It first obtains robust member models with small size throu…
With further development in the fields of computer vision, network security, natural language processing and so on so forth, deep learning technology gradually exposed certain security risks. The existing deep learning algorithms cannot effectively describe the essential characteristics of data, making the algorithm unable to give the correct result in the face of malicious input. Based on current security threats faced by deep learning, this paper introduces the problem of adversarial examples in deep learning, sorts out the existing attack and defense methods of the black box and white box, …
German Research Center for Artificial Intelligence (DFKI)（德国人工智能研究中心）；TU Kaiserslautern（凯撒斯劳滕工业大学）
The adversarial vulnerability of deep networks has spurred the interest of researchers worldwide. Unsurprisingly, like images, adversarial examples also translate to time-series data as they are an inherent weakness of the model itself rather than the modality. Several attempts have been made to defend against these adversarial attacks, particularly for the visual modality. In this paper, we perform detailed benchmarking of well-proven adversarial defense methodologies on time-series data. We restrict ourselves to the $L_{\infty}$ threat model. We also explore the trade-off between smoothness …
Since the first publication of the "OWASP Top 10" (2004), cross-site scripting (XSS) vulnerabilities have always been among the top 5 web application security bugs. Black-box vulnerability scanners are widely used in the industry to reproduce (XSS) attacks automatically. In spite of the technical sophistication and advancement, previous work showed that black-box scanners miss a non-negligible portion of vulnerabilities, and report non-existing, non-exploitable or uninteresting vulnerabilities. Unfortunately, these results hold true even for XSS vulnerabilities, which are relatively simple to …
Army Engineering University of PLA（中国人民解放军陆军工程大学）；National University of Defense Technology（国防科技大学）；University Of Science & Technology Beijing（北京科技大学）；National Computer Network and Information Security Management Center（国家计算机网络与信息安全管理中心）；Shanghai Jiao Tong University（上海交通大学）
Deep neural networks (DNNs) are increasingly being applied in malware detection and their robustness has been widely debated. Traditionally an adversarial example generation scheme relies on either detailed model information (gradient-based methods) or lots of samples to train a surrogate model, neither of which are available in most scenarios. We propose the notion of the instance-based attack. Our scheme is interpretable and can work in a black-box environment. Given a specific binary example and a malware classifier, we use the data augmentation strategies to produce enough data from which …
New York University（纽约大学）；University of Calgary（卡尔加里大学）
Human developers can produce code with cybersecurity bugs. Can emerging 'smart' code completion tools help repair those bugs? In this work, we examine the use of large language models (LLMs) for code (such as OpenAI's Codex and AI21's Jurassic J-1) for zero-shot vulnerability repair. We investigate challenges in the design of prompts that coax LLMs into generating repaired versions of insecure code. This is difficult due to the numerous ways to phrase key information - both semantically and syntactically - with natural languages. We perform a large scale study of five commercially available, b…
University of Central Florida（中佛罗里达大学）；University of Texas San Antonio（德克萨斯大学圣安东尼奥分校）
We demonstrate how a target model's generalization gap leads directly to an effective deterministic black box membership inference attack (MIA). This provides an upper bound on how secure a model can be to MIA based on a simple metric. Moreover, this attack is shown to be optimal in the expected sense given access to only certain likely obtainable metrics regarding the network's training and performance. Experimentally, this attack is shown to be comparable in accuracy to state-of-art MIAs in many cases.
University of Louisville（路易斯维尔大学）；Onycom Inc.（奥尼康公司）
The increasing scale and sophistication of cyberattacks has led to the adoption of machine learning based classification techniques, at the core of cybersecurity systems. These techniques promise scale and accuracy, which traditional rule or signature based methods cannot. However, classifiers operating in adversarial domains are vulnerable to evasion attacks by an adversary, who is capable of learning the behavior of the system by employing intelligently crafted probes. Classification accuracy in such domains provides a false sense of security, as detection can easily be evaded by carefully p…
The popularity of content management software (CMS) is growing vastly to the web developers and the business people because of its capacity for easy accessibility, manageability and usability of the distributed website contents. As per the statistics of Built with, 32% of the web applications are developed with WordPress(WP) among all other CMSs [1]. It is obvious that quite a good number of web applications were built with WP in version 4.7.0 and 4.7.1. A recent research reveals that content injection vulnerability was found available in the above two versions of WP [2]. Unauthorized content …
Mass assignment is one of the most prominent vulnerabilities in RESTful APIs. This vulnerability originates from a misconfiguration in common web frameworks, such that naming convention and automatic binding can be exploited by an attacker to craft malicious requests writing confidential resources and (massively) overriding data, that should be read-only and/or confidential. In this paper, we adopt a black-box testing perspective to automatically detect mass assignment vulnerabilities in RESTful APIs. Execution scenarios are generated purely based on the OpenAPI specification, that lists the a…
Graph Neural Networks for Vulnerability Detection: A Counterfactual Explanation
School of Computer Science and Technology, Huazhong University of Science and Technology（华中科技大学计算机科学与技术学院）；School of Electrical Engineering, Computing and Mathematical Sciences, Curtin University（科廷大学电气、计算与数学科学学院）；School of Big Data and Software Engineering, Chongqing University（重庆大学大数据与软件学院）；School of Computer Science and Engineering, University of New South Wales（新南威尔士大学计算机科学与工程学院）
Membership inference attacks seek to infer membership of individual training instances of a model to which an adversary has black-box access through a machine learning-as-a-service API. In providing an in-depth characterization of membership privacy risks against machine learning models, this paper presents a comprehensive study towards demystifying membership inference attacks from two complimentary perspectives. First, we provide a generalized formulation of the development of a black-box membership inference attack model. Second, we characterize the importance of model choice on model vulne…
Deep neural networks are susceptible to small-but-specific adversarial perturbations capable of deceiving the network. This vulnerability can lead to potentially harmful consequences in security-critical applications. To address this vulnerability, we propose a novel metric called \emph{Gradient Similarity} that allows us to capture the influence of training data on test inputs. We show that \emph{Gradient Similarity} behaves differently for normal and adversarial inputs, and enables us to detect a variety of adversarial attacks with a near perfect ROC-AUC of 95-100\%. Even white-box adversari…
The vulnerability of the high-performance machine learning models implies a security risk in applications with real-world consequences. Research on adversarial attacks is beneficial in guiding the development of machine learning models on the one hand and finding targeted defenses on the other. However, most of the adversarial attacks today leverage the gradient or logit information from the models to generate adversarial perturbation. Works in the more realistic domain: decision-based attacks, which generate adversarial perturbation solely based on observing the output label of the targeted m…
