Adv-SSL: Adversarial Self-Supervised Representation Learning with Theoretical Guarantees
School of Mathematics and Statistics, Wuhan University（武汉大学数学与统计学学院）；School of Artificial Intelligence, Wuhan University（武汉大学人工智能学院）；National Center for Applied Mathematics in Hubei, Wuhan University（湖北省应用数学中心）；Hubei Key Laboratory of Computational Science, Wuhan University（湖北省计算科学重点实验室）；Center of Statistical Research, Southwestern University of Finance and Economics（西南财经大学统计研究中心）；School of Statistics and Data Science, Southwestern University of Finance and Economics（西南财经大学统计与数据科学学院）；New Cornerstone Science Laboratory, Southwestern University of Finance and Economics（西南财经大学新基石科学实验室）；Institute for Math & AI, Wuhan University（武汉大学数学与人工智能研究院）
Chenguang Duan, Yuling Jiao, Huazhen Lin, Wensen Ma, Jerry Zhijian Yang
Learning transferable data representations from abundant unlabeled data remains a central challenge in machine learning. Although numerous self-supervised learning methods have been proposed to address this challenge, a significant class of these approaches aligns the covariance or correlation matrix with the identity matrix. Despite impressive performance across various downstream tasks, these methods often suffer from biased sample risk, leading to substantial optimization shifts in mini-batch settings and complicating theoretical analysis. In this paper, we introduce a novel \underline{\bf Adv}ersarial \underline{\bf S}elf-\underline{\bf S}upervised Representation \underline{\bf L}earning (Adv-SSL) for unbiased transfer learning with no additional cost compared to its biased counterparts. Our approach not only outperforms the existing methods across multiple benchmark datasets but is also supported by comprehensive end-to-end theoretical guarantees. Our analysis reveals that the minimax optimization in Adv-SSL encourages representations to form well-separated clusters in the embedding space, provided there is sufficient upstream unlabeled data. As a result, our method achieves strong classification performance even with limited downstream labels, shedding new light on few-shot learning.
Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning
Graduate Institute of Communication Engineering, National Taiwan University（台湾大学通信工程研究所）；Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong（香港中文大学系统工程与工程管理学系）；Shenzhen International Graduate School, Tsinghua University（清华大学深圳国际研究生院）
Zero-Sacrifice Persistent-Robustness Adversarial Defense for Pre-Trained Encoders
School of Cyber Science and Engineering, Sichuan University（四川大学网络科学与工程学院）；Key Laboratories of the Ministry of Education, Sichuan University（教育部重点实验室）；Tianfu Jiangxi Laboratory（天扶江西实验室）
Position: An Empirically Grounded Identifiability Theory Will Accelerate Self-Supervised Learning Research
Max Planck Institute for Intelligent Systems, Tübingen, Germany（马克斯·普朗克智能系统研究所）；ELLIS Institute Tübingen, Germany（图宾根ELLIS研究所）；Tübingen AI Center, Germany（图宾根人工智能中心）；Department of Computer Science, Brown University, Rhode Island, USA（布朗大学计算机科学系）；Cold Spring Harbor Laboratory, Cold Spring Harbor, New York, USA（冷泉港实验室）
HypeBoy: Generative Self-Supervised Representation Learning on Hypergraphs
Kim Jaechul Graduate School of AI（金在哲人工智能研究生院）；School of Electrical Engineering（电气工程学院）；Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Multi-task Joint Strategies of Self-supervised Representation Learning on Biomedical Networks for Drug Discovery
College of Computer Science and Electronic Engineering, Hunan University（湖南大学计算机科学与电子工程学院）；Institute of Open Source Software and Platform, Peng Cheng Laboratory（鹏城实验室开源软件与平台研究所）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；The State Key Laboratory of Chemo/Biosensing and Chemometrics, Hunan University（湖南大学化学生物传感与计量学国家重点实验室）
Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
Institute of Cyber-Systems and Control, Zhejiang University（浙江大学 Cyber-Systems 控制研究所）；Squirrel AI（松鼠AI）；Alibaba Group（阿里巴巴集团）；School of Computer Science and Technology, Zhejiang Normal University（浙江师范大学计算机科学与技术学院）；Faculty of Information Technology, Monash University（莫纳什大学信息技术学院）；Ant Group（蚂蚁集团）；Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；School of Computing and Information Systems, Singapore Management University（新加坡管理大学计算与信息系统学院）；School of Computing, University of Connecticut（康涅狄格大学计算机学院）；School of Information and Communication Technology, Griffith University（格里菲斯大学信息与通信技术学院）
Spectral regularization for adversarially-robust representation learning
John A. Paulson School of Engineering and Applied Sciences, Harvard University（哈佛大学约翰·保尔森工程与应用科学学院）；Center for Brain Science, Harvard University（哈佛大学脑科学中心）；Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University（哈佛大学肯普ner自然与人工智能研究所）
Meta AI, FAIR（元智人工智能研究院，基础人工智能研究团队）；New York University（纽约大学）；University of California, Davis（加州大学戴维斯分校）；University of Maryland（马里兰大学）；Universite de Montreal, Mila（蒙特利尔大学，米拉研究所）；Univ Gustave Eiffel, CNRS, LIGM（古斯塔夫·埃菲尔大学，法国国家科学研究中心，信息、信号与系统实验室）；Univ. Rennes, Inria, CNRS, IRISA（雷恩大学，法国国家信息与自动化研究所，法国国家科学研究中心，雷恩信息与自动化系统研究所）
A Survey of the Self Supervised Learning Mechanisms for Vision Transformers
Pattern Recognition Lab, DCIS, PIEAS（模式识别实验室，DCIS，PIEAS）；PIEAS Artificial Intelligence Center (PAIC)（PIEAS人工智能中心）；Center of Secure Cyber-Physical Security Systems, Khalifa University（安全网络物理安全系统中心，卡比拉大学）；IBM Research（IBM研究院）；Department of Computer Science（计算机科学系）；Air University（空军大学）；Department of Computer Science and Engineering（计算机科学与工程系）；Kyung Hee University (Global Campus)（庆熙大学（全球校区））；Department of Electrical Engineering and Automation（电气工程与自动化系）；Aalto University（艾尔沃斯大学）；Faculty of Engineering and Green Technology（工程与绿色技术学院）；Universiti Tunku Abdul Rahman（陶布鲁克大学）；Computer Vision Department（计算机视觉部门）；Mohamed Bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Foundation for Advancement of Science and Technology (FAST)（科学与技术进步基金会）
