Most conventional Neural Architecture Search (NAS) approaches are limited in that they only generate architectures without searching for the optimal parameters. While some NAS methods handle this issue by utilizing a supernet trained on a large-scale dataset such as ImageNet, they may be suboptimal if the target tasks are highly dissimilar from the dataset the supernet is trained on. To address such limitations, we introduce a novel problem of \emph{Neural Network Search} (NNS), whose goal is to search for the optimal pretrained network for a novel dataset and constraints (e.g. number of parameters), from a model zoo. Then, we propose a novel framework to tackle the problem, namely \emph{Task-Adaptive Neural Network Search} (TANS). Given a model-zoo that consists of network pretrained on diverse datasets, we use a novel amortized meta-learning framework to learn a cross-modal latent space with contrastive loss, to maximize the similarity between a dataset and a high-performing network on it, and minimize the similarity between irrelevant dataset-network pairs. We validate the effectiveness and efficiency of our method on ten real-world datasets, against existing NAS/AutoML baselines. The results show that our method instantly retrieves networks that outperform models obtained with the baselines with significantly fewer training steps to reach the target performance, thus minimizing the total cost of obtaining a task-optimal network. Our code and the model-zoo are available at https://github.com/wyjeong/TANS.
Hybrid-Collaborative Augmentation and Contrastive Sample Adaptive-Differential Awareness for Robust Attributed Graph Clustering
College of Information Science and Technology, Beijing University of Chemical Technology（北京化工大学信息科学与技术学院）；Beijing University of Technology（北京工业大学）；The University of Sydney Business School, The University of Sydney（悉尼大学商学院）
Graph Contrastive Learning with Adaptive Augmentation
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Alibaba Group（阿里巴巴集团）
Adaptive Multi-view Graph Contrastive Learning via Fractional-order Neural Diffusion Networks
School of Electrical and Electronic Engineering, Nanyang Technological University, Singapore（南洋理工大学电子与电气工程学院）；University College London, London, U.K.（伦敦大学学院）
Institute of Software Chinese Academy of Sciences（中国科学院软件研究所）；University of Chinese Academy of Sciences（中国科学院大学）；Defence Industry Secrecy Examination and Certification Center（国防工业保密审查认证中心）
AdNGCL: Adaptive Negative Scheduling for Graph Contrastive Learning
Jinan University（暨南大学）；University of Science and Technology of China（中国科学技术大学）；Hainan University（海南大学）；Manchester Metropolitan University（曼彻斯特城市大学）；King Abdulaziz University（阿卜杜勒阿齐兹国王大学）
University of Leeds（利兹大学）；Shaoxing University（绍兴文理学院）；Nanjing University of Information Science and Technology（南京信息工程大学）；University of Eastern Finland（东芬兰大学）；Huzhou University（湖州师范学院）
An Adaptive Contrastive Learning Model for Spike Sorting
Tsinghua Shenzhen International Graduate School（清华大学深圳国际研究生院）；Tsinghua University（清华大学）；Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）
Provably Improved Context-Based Offline Meta-RL with Attention and Contrastive Learning
Tencent AI Lab（腾讯AI实验室）；Megvii Research Shanghai（上海旷视研究院）；Tsinghua University（清华大学）；Peking University（北京大学）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）
SLSREC: Self-Supervised Contrastive Learning for Adaptive Fusion of Long- and Short-Term User Interests
School of Artificial Intelligence, Shenzhen University（深圳大学人工智能学院）；Shenzhen Technology University（深圳技术大学）；College of Computer Science, Chongqing University（重庆大学计算机学院）
Generating and Weighting Semantically Consistent Sample Pairs for Ultrasound Contrastive Learning
The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳））；Institute of Information Engineering, Chinese Academy of Sciences（中国科学院信息工程研究所）；Shenzhen Research Institute of Big Data（深圳大数据研究院）
Visual Perturbation and Adaptive Hard Negative Contrastive Learning for Compositional Reasoning in Vision-Language Models
School of Artificial Intelligence and Software Engineering, Nanyang Normal University, Henan, China（人工智能与软件工程学院，南阳师范学院，河南）；Institute for Artificial Intelligence, Peking University, Beijing, China（人工智能研究院，北京大学，北京）；Collaborative Innovation Center of Intelligent Explosion-proof Equipment, Henan, China（智能防爆设备协同创新中心，河南）
Conditional Neural Processes~(CNPs) bridge neural networks with probabilistic inference to approximate functions of Stochastic Processes under meta-learning settings. Given a batch of non-{ \it i.i.d} function instantiations, CNPs are jointly optimized for in-instantiation observation prediction and cross-instantiation meta-representation adaptation within a generative reconstruction pipeline. There can be a challenge in tying together such two targets when the distribution of function observations scales to high-dimensional and noisy spaces. Instead, noise contrastive estimation might be able…
FedTGP: Trainable Global Prototypes with Adaptive-Margin-Enhanced Contrastive Learning for Data and Model Heterogeneity in Federated Learning
Shanghai Jiao Tong University（上海交通大学）；Institute for AI Industry Research, Tsinghua University（清华大学智能产业研究院）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Queen’s University Belfast（英国女王大学）
LAMP: Learnable Meta-Path Guided Adversarial Contrastive Learning for Heterogeneous Graphs
University of New South Wales（新南威尔士大学）；Yonsei University（延世大学）；RIKEN AIP（理化学研究所人工智能项目）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
Meta-GPS++: Enhancing Graph Meta-Learning with Contrastive Learning and Self-Training
College of Computer Science and Technology, Jilin University（吉林大学计算机科学与技术学院）；University of Trento（特伦托大学）；Zhuhai College of Science and Technology（珠海科技学院）
eMargin: Revisiting Contrastive Learning with Margin-Based Separation
Department of Computer Science, Norwegian University of Science and Technology, Norway（挪威科学技术大学计算机科学系）；Department of Computer Science, United States Naval Academy, USA（美国海军学院计算机科学系）
