Over the past decades, researchers have primarily focused on improving the generalization abilities of models, with limited attention given to regulating such generalization. However, the ability of models to generalize to unintended data (e.g., harmful or unauthorized data) can be exploited by malicious adversaries in unforeseen ways, potentially resulting in violations of model ethics. Non-transferable learning (NTL), a task aimed at reshaping the generalization abilities of deep learning models, was proposed to address these challenges. While numerous methods have been proposed in this field, a comprehensive review of existing progress and a thorough analysis of current limitations remain lacking. In this paper, we bridge this gap by presenting the first comprehensive survey on NTL and introducing NTLBench, the first benchmark to evaluate NTL performance and robustness within a unified framework. Specifically, we first introduce the task settings, general framework, and criteria of NTL, followed by a summary of NTL approaches. Furthermore, we emphasize the often-overlooked issue of robustness against various attacks that can destroy the non-transferable mechanism established by NTL. Experiments conducted via NTLBench verify the limitations of existing NTL methods in robustness. Finally, we discuss the practical applications of NTL, along with its future directions and associated challenges.
When Data-Free Knowledge Distillation Meets Non-Transferable Teacher: Escaping Out-of-Distribution Trap is All You Need
Sydney AI Centre, The University of Sydney(悉尼人工智能中心,悉尼大学);School of Computer Science, Wuhan University(武汉大学计算机学院);Department of Computer Science, Hong Kong Baptist University(香港 Baptist 大学计算机系)
Transferable Adversarial Robustness for Speech Foundation Models via Hierarchical Stabilization
Sharif University of Technology(谢里夫理工大学);University of Tehran(德黑兰大学);New Uzbekistan University(新乌兹别克斯坦大学);Okinawa Institute of Science and Technology(冲绳科学技术大学院大学)
Generalized and Transferable Patient Language Representation for Phenotyping with Limited Data
The University of Texas Health Science Center at Houston(德克萨斯大学休斯顿健康科学中心);McGovern Medical School, The University of Texas Health Science Center at Houston(德克萨斯大学休斯顿健康科学中心麦戈文医学院)
Muon Learns More Robust and Transferable Features than Adam
Yale University(耶鲁大学);National University of Singapore(新加坡国立大学);University of Chinese Academy of Sciences(中国科学院大学);Academy of Mathematics and Systems Science, CAS(中国科学院数学与系统科学研究院)
本文通过鲁棒性和可迁移性视角,证明 Muon 优化器相比 Adam 和 SGD 能学习到更鲁棒、更可迁移的特征,并通过理论分析支持了经验发现。
Biologically Informed Representation Learning for Robust Cross-Center Generalization of MALDI-TOF Mass Spectrometry
Universidad Carlos III de Madrid(卡洛斯三世大学);Instituto de Investigación Sanitaria Gregorio Marañón(格雷戈里奥·马拉尼翁卫生研究所);Hospital General Universitario Gregorio Marañón(格雷戈里奥·马拉尼翁大学总医院)
DP-TRAE: A Dual-Phase Merging Transferable Reversible Adversarial Example for Image Privacy Protection
School of Computer and Information Engineering, Xiamen University of Technology(厦门理工大学计算机与信息工程学院);School of Computer Science, Engineering Research Center of Machine Learning and Industry Intelligence, Sichuan University(四川大学计算机科学学院,机器学习与产业智能工程研究中心);Department of Computer and Information Science, Faculty of Science and Technology, University of Macau(澳门大学科学与技术学院计算机与信息科学系);Department of Electrical and Electronic Engineering, University of Hong Kong(香港大学电子与电气工程系);Institute of Space Internet, Fudan University(复旦大学空间互联网研究所);School of Computer Science and Engineering, Nanyang Technological University(南洋理工大学计算机科学与工程学院)
Representation Learning for Tabular Data: A Comprehensive Survey
School of Artificial Intelligence, Nanjing University(人工智能学院,南京大学);National Key Laboratory for Novel Software Technology, Nanjing University(新型软件技术国家实验室,南京大学)