Modern AI systems are reaping the advantage of novel learning methods. With their increasing usage, we are realizing the limitations and shortfalls of these systems. Brittleness to minor adversarial changes in the input data, ability to explain the decisions, address the bias in their training data, high opacity in terms of revealing the lineage of the system, how they were trained and tested, and under which parameters and conditions they can reliably guarantee a certain level of performance, are some of the most prominent limitations. Ensuring the privacy and security of the data, assigning appropriate credits to data sources, and delivering decent outputs are also required features of an AI system. We propose the tutorial on Trustworthy AI to address six critical issues in enhancing user and public trust in AI systems, namely: (i) bias and fairness, (ii) explainability, (iii) robust mitigation of adversarial attacks, (iv) improved privacy and security in model building, (v) being decent, and (vi) model attribution, including the right level of credit assignment to the data sources, model architectures, and transparency in lineage.
Siemens AG（西门子股份公司）；Technical University of Munich（慕尼黑工业大学）
To facilitate a wide-spread acceptance of AI systems guiding decision making in real-world applications, trustworthiness of deployed models is key. That is, it is crucial for predictive models to be uncertainty-aware and yield well-calibrated (and thus trustworthy) predictions for both in-domain samples as well as under domain shift. Recent efforts to account for predictive uncertainty include post-processing steps for trained neural networks, Bayesian neural networks as well as alternative non-Bayesian approaches such as ensemble approaches and evidential deep learning. Here, we propose an ef…
Toward Trustworthy AI: Multi-Target Adversarial Attacks and Robust Defenses for Continuous Data Summarization
Nankai University（南开大学）；James Cook University（詹姆斯库克大学）；Western Sydney University（西悉尼大学）；Beijing University of Technology（北京工业大学）；Fuzhou University（福州大学）；Nanjing University of Science and Technology（南京理工大学）；CSIRO's Data 61（澳大利亚联邦科学与工业研究组织Data61）；The University of Adelaide（阿德莱德大学）
Ensuring trustworthiness in machine learning (ML) models is a multi-dimensional task. In addition to the traditional notion of predictive performance, other notions such as privacy, fairness, robustness to distribution shift, adversarial robustness, interpretability, explainability, and uncertainty quantification are important considerations to evaluate and improve (if deficient). However, these sub-disciplines or 'pillars' of trustworthiness have largely developed independently, which has limited us from understanding their interactions in real-world ML pipelines. In this paper, focusing spec…
SUPER-Net: Trustworthy Image Segmentation via Uncertainty Propagation in Encoder-Decoder Networks
Department of Electrical and Computer Engineering, Rowan University（电气与计算机工程系，罗文大学）；Chester F. Carlson Center for Imaging Science, Rochester Institute of Technology（成像科学中心，罗切斯特理工学院）；Department of Neurology, University of Alabama at Birmingham School of Medicine（神经病学系，阿拉巴马大学伯明翰医学学院）；Machine Learning Department, Moffit Cancer Center（机器学习系，莫菲特癌症中心）
On the Robustness of Adversarial Training Against Uncertainty Attacks
Department of Electric and Electronic Engineering, University of Cagliari（卡利亚里大学电子与电气工程系）；Department of Informatics, Bioengineering, Robotics, and Systems Engineering, University of Genova（热那亚大学信息学、生物工程、机器人与系统工程系）
Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs
MATS ELLIS Institute（MATS ELLIS研究所）；Max Planck Institute for Intelligent Systems（智能系统马克斯·普朗克研究所）；Tübingen AI Center（图宾根人工智能中心）；Imperial College London（伦敦帝国理工学院）
University of Georgia（佐治亚大学）；Tencent AI Seattle Lab（腾讯AI西雅图实验室）；Microsoft Research（微软研究院）；University of Waterloo（滑铁卢大学）；Hong Kong Polytechnic University（香港理工大学）
Research Group on Intelligent Engineering and Computing for Advanced Innovation and Development (GECAD), School of Engineering, Polytechnic of Porto (ISEP/IPP)（先进创新与发展智能工程与计算研究组（GECAD），波尔图理工学院工程学院（ISEP/IPP））
The Internet of Things (IoT) faces tremendous security challenges. Machine learning models can be used to tackle the growing number of cyber-attack variations targeting IoT systems, but the increasing threat posed by adversarial attacks restates the need for reliable defense strategies. This work describes the types of constraints required for a realistic adversarial cyber-attack example and proposes a methodology for a trustworthy adversarial robustness analysis with a realistic adversarial evasion attack vector. The proposed methodology was used to evaluate three supervised algorithms, Rando…
Reliable Disentanglement Multi-view Learning Against View Adversarial Attacks
College of Computer Science, Sichuan University（四川大学计算机学院）；Laboratory of Intelligent Collaborative Computing, University of Electronic Science and Technology of China（电子科技大学智能协同计算实验室）；National Key Laboratory of Fundamental Algorithms and Models for Engineering Numerical Simulation, Sichuan University（四川省工程数值模拟基础算法与模型重点实验室）；Tianfu Jincheng Laboratory（天府锦城实验室）；Sichuan National Innovation New Vision UHD Video Technology Co., Ltd.（四川省创新新视野UHD视频技术有限公司）
In the recent quest for trustworthy neural networks, we present Spiking Neural Network (SNN) as a potential candidate for inherent robustness against adversarial attacks. In this work, we demonstrate that adversarial accuracy of SNNs under gradient-based attacks is higher than their non-spiking counterparts for CIFAR datasets on deep VGG and ResNet architectures, particularly in blackbox attack scenario. We attribute this robustness to two fundamental characteristics of SNNs and analyze their effects. First, we exhibit that input discretization introduced by the Poisson encoder improves advers…
National University of Singapore（新加坡国立大学）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；The University of Queensland（昆士兰大学）；Griffith University（格里菲斯大学）
Neural networks have been widely applied in security applications such as spam and phishing detection, intrusion prevention, and malware detection. This black-box method, however, often has uncertainty and poor explainability in applications. Furthermore, neural networks themselves are often vulnerable to adversarial attacks. For those reasons, there is a high demand for trustworthy and rigorous methods to verify the robustness of neural network models. Adversarial robustness, which concerns the reliability of a neural network when dealing with maliciously manipulated inputs, is one of the hot…
University of Liverpool（利物浦大学）；University of Oxford（牛津大学）；Lancaster University（兰卡斯特大学）；Defence Science and Technology Laboratory (Dstl)（国防科学技术实验室）；Queen’s University Belfast（贝尔法斯特女王大学）
In the past few years, significant progress has been made on deep neural networks (DNNs) in achieving human-level performance on several long-standing tasks. With the broader deployment of DNNs on various applications, the concerns over their safety and trustworthiness have been raised in public, especially after the widely reported fatal incidents involving self-driving cars. Research to address these concerns is particularly active, with a significant number of papers released in the past few years. This survey paper conducts a review of the current research effort into making DNNs safe and …
Enhancing model robustness under new and even adversarial environments is a crucial milestone toward building trustworthy machine learning systems. Current robust training methods such as adversarial training explicitly uses an "attack" (e.g., $\ell_{\infty}$-norm bounded perturbation) to generate adversarial examples during model training for improving adversarial robustness. In this paper, we take a different perspective and propose a new framework called SPROUT, self-progressing robust training. During model training, SPROUT progressively adjusts training label distribution via our proposed…
Towards Trustworthy and Aligned Machine Learning: A Data-centric Survey with Causality Perspectives
Bundelkhand Institute of Engineering and Technology（本德尔坎德工程技术学院）；Nanyang Technological University（南洋理工大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
IBM Research（IBM研究院）；Univ Lyon（里昂大学）；UJM-Saint-Etienne（圣艾蒂安让·莫内大学）；CNRS（法国国家科学研究中心）；Institut d Optique Graduate School（高等光学学校）
Adversarial Robustness Toolbox (ART) is a Python library supporting developers and researchers in defending Machine Learning models (Deep Neural Networks, Gradient Boosted Decision Trees, Support Vector Machines, Random Forests, Logistic Regression, Gaussian Processes, Decision Trees, Scikit-learn Pipelines, etc.) against adversarial threats and helps making AI systems more secure and trustworthy. Machine Learning models are vulnerable to adversarial examples, which are inputs (images, texts, tabular data, etc.) deliberately modified to produce a desired response by the Machine Learning model.…
IEMN CNRS-UMR8520（法国国家科学研究中心电子、微电子与纳米技术研究所 UMR8520）；Université Polytechnique Hauts-De-France（上法兰西理工大学）；Institute of Computer Engineering, Technische Universität Wien（维也纳技术大学计算机工程研究所）；New York University Abu Dhabi（纽约大学阿布扎比分校）
Deep Learning (DL) algorithms have gained popularity owing to their practical problem-solving capacity. However, they suffer from a serious integrity threat, i.e., their vulnerability to adversarial attacks. In the quest for DL trustworthiness, recent works claimed the inherent robustness of Spiking Neural Networks (SNNs) to these attacks, without considering the variability in their structural spiking parameters. This paper explores the security enhancement of SNNs through internal structural parameters. Specifically, we investigate the SNNs robustness to adversarial attacks with different va…
Although Deep Neural Networks (DNNs) have shown incredible performance in perceptive and control tasks, several trustworthy issues are still open. One of the most discussed topics is the existence of adversarial perturbations, which has opened an interesting research line on provable techniques capable of quantifying the robustness of a given input. In this regard, the Euclidean distance of the input from the classification boundary denotes a well-proved robustness assessment as the minimal affordable adversarial perturbation. Unfortunately, computing such a distance is highly complex due the …
National Taiwan University（国立台湾大学）；Mobile Drive Technology（移动驱动科技公司）
Fairness and robustness play vital roles in trustworthy machine learning. Observing safety-critical needs in various annotation-expensive vision applications, we introduce a novel learning framework, Fair Robust Active Learning (FRAL), generalizing conventional active learning to fair and adversarial robust scenarios. This framework allows us to achieve standard and robust minimax fairness with limited acquired labels. In FRAL, we then observe existing fairness-aware data selection strategies suffer from either ineffectiveness under severe data imbalance or inefficiency due to huge computation…
Adversarial Robustness in Two-Stage Learning-to-Defer: Algorithms and Guarantees
School of Computing, National University of Singapore, Singapore（新加坡国立大学计算机学院）；IRIT, Université de Toulouse, CNRS, Toulouse INP, Toulouse, France（图卢兹大学IRIT研究所、法国国家科学研究中心、图卢兹INP）；Institute for Infocomm Research, Agency for Science, Technology and Research, Singapore（新加坡科技研究局信息与通信研究所）
Enhancing Adversarial Robustness with Conformal Prediction: A Framework for Guaranteed Model Reliability
Huaiyin Institute of Technology, Huai'an, China（淮阴职业技术学院）；Chengdu Research Institute, City University of Hong Kong, Chengdu, China（香港城市大学成都研究院）；Saarland University, Saarbrücken, Germany（萨尔兰大学）；Alpha Benito Research, Los Angeles, USA（Alpha Benito研究公司）；City University of Hong Kong, Hong Kong, China（香港城市大学）
Adversarial Attacks and Detection in Visual Place Recognition for Safer Robot Navigation
QUT Centre for Robotics（昆士兰科技大学机器人中心）；School of Electrical Engineering and Robotics at the Queensland University of Technology（昆士兰科技大学电气工程与机器人学院）；CIICADA Lab（CIICADA实验室）
Princeton University（普林斯顿大学）；Shanghai Jiaotong University（上海交通大学）；University of Pennsylvania（宾夕法尼亚大学）；RIKEN AIP（理化学研究所先进智能项目中心）；University of Tokyo（东京大学）
Fairness and accountability are two essential pillars for trustworthy Artificial Intelligence (AI) in healthcare. However, the existing AI model may be biased in its decision marking. To tackle this issue, we propose an adversarial multi-task training strategy to simultaneously mitigate and detect bias in the deep learning-based medical image analysis system. Specifically, we propose to add a discrimination module against bias and a critical module that predicts unfairness within the base classification model. We further impose an orthogonality regularization to force the two modules to be ind…
