As machine learning (ML) systems become pervasive, safeguarding their security is critical. However, recently it has been demonstrated that motivated adversaries are able to mislead ML systems by perturbing test data using semantic transformations. While there exists a rich body of research providing provable robustness guarantees for ML models against $\ell_p$ norm bounded adversarial perturbations, guarantees against semantic perturbations remain largely underexplored. In this paper, we provide TSS -- a unified framework for certifying ML robustness against general adversarial semantic transformations. First, depending on the properties of each transformation, we divide common transformations into two categories, namely resolvable (e.g., Gaussian blur) and differentially resolvable (e.g., rotation) transformations. For the former, we propose transformation-specific randomized smoothing strategies and obtain strong robustness certification. The latter category covers transformations that involve interpolation errors, and we propose a novel approach based on stratified sampling to certify the robustness. Our framework TSS leverages these certification strategies and combines with consistency-enhanced training to provide rigorous certification of robustness. We conduct extensive experiments on over ten types of challenging semantic transformations and show that TSS significantly outperforms the state of the art. Moreover, to the best of our knowledge, TSS is the first approach that achieves nontrivial certified robustness on the large-scale ImageNet dataset. For instance, our framework achieves 30.4% certified robust accuracy against rotation attack (within $\pm 30^\circ$) on ImageNet. Moreover, to consider a broader range of transformations, we show TSS is also robust against adaptive attacks and unforeseen image corruptions such as CIFAR-10-C and ImageNet-C.
Mila（米拉）；University of Texas at Austin（得克萨斯大学奥斯汀分校）；Peking University（北京大学）
Randomized classifiers have been shown to provide a promising approach for achieving certified robustness against adversarial attacks in deep learning. However, most existing methods only leverage Gaussian smoothing noise and only work for $\ell_2$ perturbation. We propose a general framework of adversarial certification with non-Gaussian noise and for more general types of attacks, from a unified functional optimization perspective. Our new framework allows us to identify a key trade-off between accuracy and robustness via designing smoothing distributions, helping to design new families of n…
We present a new certification method for image and point cloud segmentation based on randomized smoothing. The method leverages a novel scalable algorithm for prediction and certification that correctly accounts for multiple testing, necessary for ensuring statistical guarantees. The key to our approach is reliance on established multiple-testing correction mechanisms as well as the ability to abstain from classifying single pixels or points while still robustly segmenting the overall input. Our experimental evaluation on synthetic data and challenging datasets, such as Pascal Context, Citysc…
Carnegie Mellon University（卡内基梅隆大学）；University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
A vast literature shows that the learning-based visual perception model is sensitive to adversarial noises, but few works consider the robustness of robotic perception models under widely-existing camera motion perturbations. To this end, we study the robustness of the visual perception model under camera motion perturbations to investigate the influence of camera motion on robotic perception. Specifically, we propose a motion smoothing technique for arbitrary image classification models, whose robustness under camera motion perturbations could be certified. The proposed robustness certificati…
MIT（麻省理工学院）；MIT-IBM Watson AI Lab, IBM Research（麻省理工学院-IBM Watson人工智能实验室，IBM研究）
Randomized smoothing is a recently proposed defense against adversarial attacks that has achieved SOTA provable robustness against $\ell_2$ perturbations. A number of publications have extended the guarantees to other metrics, such as $\ell_1$ or $\ell_\infty$, by using different smoothing measures. Although the current framework has been shown to yield near-optimal $\ell_p$ radii, the total safety region certified by the current framework can be arbitrarily small compared to the optimal. In this work, we propose a framework to improve the certified safety region for these smoothed classifiers…
Institut de Recherche Technologique SystemX（SystemX技术研究院）；Univ. Bordeaux（波尔多大学）；CNRS（法国国家科学研究中心）；Bordeaux INP（波尔多国立理工学院）；IMB（波尔多数学研究所）
Certification of neural networks is an important and challenging problem that has been attracting the attention of the machine learning community since few years. In this paper, we focus on randomized smoothing (RS) which is considered as the state-of-the-art method to obtain certifiably robust neural networks. In particular, a new data-dependent RS technique called ANCER introduced recently can be used to certify ellipses with orthogonal axis near each input data of the neural network. In this work, we remark that ANCER is not invariant under rotation of input data and propose a new rotationa…
Randomized smoothing, a method to certify a classifier's decision on an input is invariant under adversarial noise, offers attractive advantages over other certification methods. It operates in a black-box and so certification is not constrained by the size of the classifier's architecture. Here, we extend the work of Li et al. \cite{li2018second}, studying how the choice of divergence between smoothing measures affects the final robustness guarantee, and how the choice of smoothing measure itself can lead to guarantees in differing threat models. To this end, we develop a method to certify ro…
University of Oxford（牛津大学）；Five AI Ltd.（Five AI有限公司）；King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学（KAUST））
Randomized smoothing has recently emerged as an effective tool that enables certification of deep neural network classifiers at scale. All prior art on randomized smoothing has focused on isotropic $\ell_p$ certification, which has the advantage of yielding certificates that can be easily compared among isotropic methods via $\ell_p$-norm radius. However, isotropic certification limits the region that can be certified around an input to worst-case adversaries, i.e., it cannot reason about other "close", potentially large, constant prediction safe regions. To alleviate this issue, (i) we theore…
King Abdullah University of Science and Technology (KAUST)（阿卜杜拉国王科技大学）；University of Oxford（牛津大学）
Randomized smoothing is a recent technique that achieves state-of-art performance in training certifiably robust deep neural networks. While the smoothing family of distributions is often connected to the choice of the norm used for certification, the parameters of these distributions are always set as global hyper parameters independent from the input data on which a network is certified. In this work, we revisit Gaussian randomized smoothing and show that the variance of the Gaussian distribution can be optimized at each input so as to maximize the certification radius for the construction o…
Institute of Information Science, Academia Sinica（中央研究院资讯科学研究所）
Randomized smoothing has established state-of-the-art provable robustness against $\ell_2$ norm adversarial attacks with high probability. However, the introduced Gaussian data augmentation causes a severe decrease in natural accuracy. We come up with a question, "Is it possible to construct a smoothed classifier without randomization while maintaining natural accuracy?". We find the answer is definitely yes. We study how to transform any classifier into a certified robust classifier based on a popular and elegant mathematical tool, Bernstein polynomial. Our method provides a deterministic alg…
University of Connecticut（康涅狄格大学）；Illinois Institute of Technology（伊利诺伊理工学院）
We study certified robustness of machine learning classifiers against adversarial perturbations. In particular, we propose the first universally approximated certified robustness (UniCR) framework, which can approximate the robustness certification of any input on any classifier against any $\ell_p$ perturbations with noise generated by any continuous probability distribution. Compared with the state-of-the-art certified defenses, UniCR provides many significant benefits: (1) the first universal robustness certification framework for the above 4 'any's; (2) automatic robustness certification t…
We extend randomized smoothing to cover parameterized transformations (e.g., rotations, translations) and certify robustness in the parameter space (e.g., rotation angle). This is particularly challenging as interpolation and rounding effects mean that image transformations do not compose, in turn preventing direct certification of the perturbed image (unlike certification with $\ell^p$ norms). We address this challenge by introducing three different kinds of defenses, each with a different guarantee (heuristic, distributional and individual) stemming from the method used to bound the interpol…
Randomized smoothing is the dominant standard for provable defenses against adversarial examples. Nevertheless, this method has recently been proven to suffer from important information theoretic limitations. In this paper, we argue that these limitations are not intrinsic, but merely a byproduct of current certification methods. We first show that these certificates use too little information about the classifier, and are in particular blind to the local curvature of the decision boundary. This leads to severely sub-optimal robustness guarantees as the dimension of the problem increases. We t…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Washington University in St. Louis（圣路易斯华盛顿大学）；Carnegie Mellon University（卡内基梅隆大学）
As reinforcement learning (RL) has achieved great success and been even adopted in safety-critical domains such as autonomous vehicles, a range of empirical studies have been conducted to improve its robustness against adversarial attacks. However, how to certify its robustness with theoretical guarantees still remains challenging. In this paper, we present the first unified framework CROP (Certifying Robust Policies for RL) to provide robustness certification on both action and reward levels. In particular, we propose two robustness certification criteria: robustness of per-state actions and …
Randomized smoothing is a recent and celebrated solution to certify the robustness of any classifier. While it indeed provides a theoretical robustness against adversarial attacks, the dimensionality of current classifiers necessarily imposes Monte Carlo approaches for its application in practice. This paper questions the effectiveness of randomized smoothing as a defense, against state of the art black-box attacks. This is a novel perspective, as previous research works considered the certification as an unquestionable guarantee. We first formally highlight the mismatch between a theoretical …
Effects of Exponential Gaussian Distribution on (Double Sampling) Randomized Smoothing
Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）；CSIRO’s Data61（联邦科学与工业研究组织数据61部）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Simon Fraser University（西蒙菲莎大学）；University of Chicago（芝加哥大学）
Cooperative multi-agent reinforcement learning (c-MARL) is widely applied in safety-critical scenarios, thus the analysis of robustness for c-MARL models is profoundly important. However, robustness certification for c-MARLs has not yet been explored in the community. In this paper, we propose a novel certification method, which is the first work to leverage a scalable approach for c-MARLs to determine actions with guaranteed certified bounds. c-MARL certification poses two key challenges compared with single-agent systems: (i) the accumulated uncertainty as the number of agents increases; (ii…
Cert-SSBD: Certified Backdoor Defense with Sample-Specific Smoothing Noises
School of Control and Computer Engineering, North China Electric Power University（控制与计算机工程学院，华北电力大学）；Research Institute, China Unicom（中国unicom研究院）；National Engineering Research Center of Next Generation Internet Broadband Service Application（下一代互联网宽带服务应用国家工程研究中心）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）
TPC: Transformation-Specific Smoothing for Point Cloud Models
Institute for Interdisciplinary Information Sciences, Tsinghua University（清华大学交叉信息研究院）；University of Illinois Urbana-Champaign (UIUC)（伊利诺伊大学厄巴纳-香槟分校（UIUC））
Laplace-Bridged Randomized Smoothing for Fast Certified Robustness
Department of Computer Science, Old Dominion University, Norfolk, VA, USA（旧 Dominion 大学计算机科学系）；Department of Mathematical Sciences, University of Texas at El Paso, El Paso, TX, USA（德克萨斯理工大学数学科学系）；School of Cybersecurity, Old Dominion University, Norfolk, VA, USA（旧 Dominion 大学网络安全学院）
Cornell University（康奈尔大学）；Alibaba Group Inc.（阿里巴巴集团）
Recently smoothing deep neural network based classifiers via isotropic Gaussian perturbation is shown to be an effective and scalable way to provide state-of-the-art probabilistic robustness guarantee against $\ell_2$ norm bounded adversarial perturbations. However, how to train a good base classifier that is accurate and robust when smoothed has not been fully investigated. In this work, we derive a new regularized risk, in which the regularizer can adaptively encourage the accuracy and robustness of the smoothed counterpart when training the base classifier. It is computationally efficient a…
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Peking University（北京大学）
Neural networks (NNs) are known to be vulnerable against adversarial perturbations, and thus there is a line of work aiming to provide robustness certification for NNs, such as randomized smoothing, which samples smoothing noises from a certain distribution to certify the robustness for a smoothed classifier. However, as shown by previous work, the certified robust radius in randomized smoothing suffers from scaling to large datasets ("curse of dimensionality"). To overcome this hurdle, we propose a Double Sampling Randomized Smoothing (DSRS) framework, which exploits the sampled probability f…
School of Computation, Information & Technology, Technical Univ. of Munich（慕尼黑工业大学计算、信息与技术学院）；Fraunhofer Institute for Cognitive Systems IKS（弗劳恩霍夫认知系统研究所IKS）
Patch adversarial attacks on images, in which the attacker can distort pixels within a region of bounded size, are an important threat model since they provide a quantitative model for physical adversarial attacks. In this paper, we introduce a certifiable defense against patch attacks that guarantees for a given image and patch attack size, no patch adversarial examples exist. Our method is related to the broad class of randomized smoothing robustness schemes which provide high-confidence probabilistic robustness certificates. By exploiting the fact that patch attacks are more constrained tha…
Randomized smoothing is a general technique for computing sample-dependent robustness guarantees against adversarial attacks for deep classifiers. Prior works on randomized smoothing against L_1 adversarial attacks use additive smoothing noise and provide probabilistic robustness guarantees. In this work, we propose a non-additive and deterministic smoothing method, Deterministic Smoothing with Splitting Noise (DSSN). To develop DSSN, we first develop SSN, a randomized method which involves generating each noisy smoothing sample by first randomly splitting the input space and then returning a …
Defence Science and Technology Group（国防科学技术集团）；University of Melbourne（墨尔本大学）
In response to subtle adversarial examples flipping classifications of neural network models, recent research has promoted certified robustness as a solution. There, invariance of predictions to all norm-bounded attacks is achieved through randomised smoothing of network inputs. Today's state-of-the-art certifications make optimal use of the class output scores at the input instance under test: no better radius of certification (under the $L_2$ norm) is possible given only these score. However, it is an open question as to whether such lower bounds can be improved using local information aroun…
Tsinghua University（清华大学）；RealAI（瑞莱智慧）；Peng Cheng Laboratory（鹏城实验室）；Tsinghua University-China Mobile Communications Group Co., Ltd. Joint Institute（清华大学-中国移动通信集团有限公司联合研究院）；Research Institute of Tsinghua University in Shenzhen（深圳清华大学研究院）
Certified defenses such as randomized smoothing have shown promise towards building reliable machine learning systems against $\ell_p$-norm bounded attacks. However, existing methods are insufficient or unable to provably defend against semantic transformations, especially those without closed-form expressions (such as defocus blur and pixelate), which are more common in practice and often unrestricted. To fill up this gap, we propose generalized randomized smoothing (GSmooth), a unified theoretical framework for certifying robustness against general semantic transformations via a novel dimens…
Certified Robustness for Large Language Models with Self-Denoising
UC Santa Barbara（加州大学圣巴巴拉分校）；The Hong Kong Polytechnic University（香港理工大学）；Michigan State University（密歇根州立大学）；MIT-IBM Watson AI Lab（麻省理工学院-IBM沃森人工智能实验室）
Carnegie Mellon University（卡内基梅隆大学）；Bosch Center for AI（博世人工智能中心）
Machine learning algorithms are known to be susceptible to data poisoning attacks, where an adversary manipulates the training data to degrade performance of the resulting classifier. In this work, we present a unifying view of randomized smoothing over arbitrary functions, and we leverage this novel characterization to propose a new strategy for building classifiers that are pointwise-certifiably robust to general data poisoning attacks. As a specific instantiation, we utilize our framework to build linear classifiers that are robust to a strong variant of label flipping, where each test exam…
National University of Singapore（新加坡国立大学）；Technical University of Munich（慕尼黑工业大学）
The current state-of-the-art defense methods against adversarial examples typically focus on improving either empirical or certified robustness. Among them, adversarially trained (AT) models produce empirical state-of-the-art defense against adversarial examples without providing any robustness guarantees for large classifiers or higher-dimensional inputs. In contrast, existing randomized smoothing based models achieve state-of-the-art certified robustness while significantly degrading the empirical robustness against adversarial examples. In this paper, we propose a novel method, called \emph…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Zhejiang University（浙江大学）；IBM Research（IBM研究院）
Federated Learning (FL) as a distributed learning paradigm that aggregates information from diverse clients to train a shared global model, has demonstrated great success. However, malicious clients can perform poisoning attacks and model replacement to introduce backdoors into the trained global model. Although there have been intensive studies designing robust aggregation methods and empirical robust federated training protocols against backdoors, existing approaches lack robustness certification. This paper provides the first general framework, Certifiably Robust Federated Learning (CRFL), …
Center for Data Science, Peking University（北京大学数据科学中心）；School of Mathematical Sciences, Peking University（北京大学数学科学学院）；National School of Development, Peking University（北京大学国家发展研究院）；Beijing International Center for Mathematical Research（北京国际数学研究中心）；Institute for Artificial Inteligence, Peking University（北京大学人工智能研究所）
Randomized smoothing has achieved state-of-the-art certified robustness against $l_2$-norm adversarial attacks. However, it is not wholly resolved on how to find the optimal base classifier for randomized smoothing. In this work, we employ a Smoothed WEighted ENsembling (SWEEN) scheme to improve the performance of randomized smoothed classifiers. We show the ensembling generality that SWEEN can help achieve optimal certified robustness. Furthermore, theoretical analysis proves that the optimal SWEEN model can be obtained from training under mild assumptions. We also develop an adaptive predict…
National Technical University of Athens（希腊雅典国家技术大学）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；CSX-AI（CSX-AI公司）；Carl von Ossietzky University of Oldenburg（奥尔登堡卡尔·冯·奥西特齐大学）；Chalmers University of Technology（查尔姆斯理工大学）
Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy
Department of Computer Science, Technical University of Munich（慕尼黑技术大学计算机科学系）；Munich Data Science Institute（慕尼黑数据科学研究所）；MCML；Machine Learning Research, Morgan Stanley（摩根大通机器学习研究）
