Self-Supervised Learning (SSL) has shown great promise in learning representations from unlabeled data. The power of learning representations without the need for human annotations has made SSL a widely used technique in real-world problems. However, SSL methods have recently been shown to be vulnerable to backdoor attacks, where the learned model can be exploited by adversaries to manipulate the learned representations, either through tampering the training data distribution, or via modifying the model itself. This work aims to address defending against backdoor attacks in SSL, where the adversary has access to a realistic fraction of the SSL training data, and no access to the model. We use novel methods that are computationally efficient as well as generalizable across different problem settings. We also investigate the adversarial robustness of SSL models when trained with our method, and show insights into increased robustness in SSL via frequency domain augmentations. We demonstrate the effectiveness of our method on a variety of SSL benchmarks, and show that our method is able to mitigate backdoor attacks while maintaining high performance on downstream tasks. Code for our work is available at github.com/Aryan-Satpathy/Backdoor
Invisible Backdoor Attack against Self-supervised Learning
Zhejiang University(浙江大学);Rutgers University(罗格斯大学);Nanyang Technological University(南洋理工大学);Nanjing University(南京大学);New Jersey Institute of Technology(新泽西理工学院);University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校)
GhostEncoder: Stealthy Backdoor Attacks with Dynamic Triggers to Pre-trained Encoders in Self-supervised Learning
Nanjing University of Aeronautics and Astronautics(南京航空航天大学);Nanjing University of Aeronautics and Astronautics Shenzhen Research Institute(南京航空航天大学深圳研究院);Zhejiang Lab(之江实验室);Wuhan University(武汉大学);Xi’an Jiaotong University(西安交通大学)
DEFUSE: Generalizable Backdoor Defense for Self-Supervised Encoders with Generative Priors
Ant Group(蚂蚁集团);Purple Mountain Laboratories(紫金山实验室);City University of Hong Kong(香港城市大学);Beijing Institute of Technology(北京理工大学);Alibaba Group(阿里巴巴集团)
Erasing Self-Supervised Learning Backdoor by Cluster Activation Masking
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所 多模态人工智能系统国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Xiaomi Inc.(小米公司);State Key Laboratory of CNS/ATM, Beihang University(北京航空航天大学 CNS/ATM 国家重点实验室);College of Computer Science, Inner Mongolia University(内蒙古大学 计算机学院)
BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning
National University of Singapore(新加坡国立大学);The Chinese University of Hong Kong, Shenzhen(香港中文大学(深圳));Beihang University(北京航空航天大学);Sun Yat-sen University-Shenzhen(中山大学(深圳))
Steering LVLMs via Sparse Autoencoder for Hallucination Mitigation
School of Computer Science and Engineering, Southeast University(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University)(东南大学新一代人工智能技术及其交叉应用关键实验室);Foundation Model Research Center, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所基础模型研究中心);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Wuhan University of Technology(武汉理工大学);National University of Singapore(新加坡国立大学)
SSL-SoilNet: A Hybrid Transformer-based Framework with Self-Supervised Learning for Large-scale Soil Organic Carbon Prediction
University of Tübingen(蒂宾根大学);University of Tehran(德黑兰大学);Simon Fraser University(西蒙菲莎大学);Stuttgart University of Applied Sciences(斯图加特应用技术大学);University of Florida(佛罗里达大学)