Bosch Center for AI（博世人工智能中心）；University of Tübingen（蒂宾根大学）；University of Mannheim（曼海姆大学）；Max Planck Institute for Informatics（马克斯·普朗克信息学研究所）；Bosch Center for Artificial Intelligence（博世人工智能中心）
Ugur Ali Kaplan, Margret Keuper, Anna Khoreva, Dan Zhang, Yumeng Li
备注：
Accepted at ICML 2024 Workshop on Foundation Models in the Wild
Foundation models (FMs) have revolutionized computer vision, enabling effective learning across different domains. However, their performance under domain shift is yet underexplored. This paper investigates the zero-shot domain adaptation potential of FMs by comparing different backbone architectures and introducing novel domain-aware components that leverage domain related textual embeddings. We propose domain adaptive normalization, termed as Domino, which explicitly leverages domain embeddings during fine-tuning, thus making the model domain aware. Ultimately, Domino enables more robust computer vision models that can adapt effectively to various unseen domains.
Transfer learning is important for foundation models to adapt to downstream tasks. However, many foundation models are proprietary, so users must share their data with model owners to fine-tune the models, which is costly and raise privacy concerns. Moreover, fine-tuning large foundation models is computation-intensive and impractical for most downstream users. In this paper, we propose Offsite-Tuning, a privacy-preserving and efficient transfer learning framework that can adapt billion-parameter foundation models to downstream data without access to the full model. In offsite-tuning, the mode…
FedPEAT: Convergence of Federated Learning, Parameter-Efficient Fine Tuning, and Emulator Assisted Tuning for Artificial Intelligence Foundation Models with Mobile Edge Computing
Decision-Focused Fine-Tuning of Time Series Foundation Models for Dispatchable Feeder Optimization
Institute for Automation and Applied Informatics (IAI), Karlsruhe Institute of Technology (KIT)（自动化与应用信息技术研究所（IAI），卡尔斯鲁厄理工学院）；Institute of Biological and Chemical Systems (IBCS), Karlsruhe Institute of Technology (KIT)（生物与化学系统研究所（IBCS），卡尔斯鲁厄理工学院）；Scientific Computing Center (SCC), Karlsruhe Institute of Technology (KIT)（科学计算中心（SCC），卡尔斯鲁厄理工学院）
Are Large Brainwave Foundation Models Capable Yet? Insights from Fine-tuning
Imperial College London（伦敦帝国学院）；Archimedes / Athena Research Unit（阿基米德/雅典娜研究单位）；Aristotle University of Thessaloniki（雅典娜大学）；Kapodistrian University of Athens（雅典kapodistrian大学）
Dynamic Gradient Sparsification Training for Few-Shot Fine-tuning of CT Lymph Node Segmentation Foundation Model
Sichuan Cancer Hospital and Institute, University of Electronic Science and Technology of China（电子科技大学附属四川省肿瘤医院）；School of Mechanical and Electrical Engineering, University of Electronic Science and Technology of China（电子科技大学机械与电气工程学院）；The Chinese University of Hong Kong（香港中文大学）；Shanghai AI Laboratory（上海人工智能实验室）；Stanford University School of Medicine（斯坦福大学医学院）
Self-supervised pre-training of a speech foundation model, followed by supervised fine-tuning, has shown impressive quality improvements on automatic speech recognition (ASR) tasks. Fine-tuning separate foundation models for many downstream tasks are expensive since the foundation model is usually very big. Parameter-efficient fine-tuning methods (e.g. adapter, sparse update methods) offer an alternative paradigm where a small set of parameters are updated to adapt the foundation model to new tasks. However, these methods still suffer from a high computational memory cost and slow training spe…
GenFT: A Generative Parameter-Efficient Fine-Tuning Method for Pretrained Foundation Models
Department of Mathematics, Hong Kong Baptist University, Hong Kong, China（香港 Baptist 大学数学系，香港，中国）；Department of Computer Science, Harbin Institute of Technology, Shenzhen, China（哈尔滨工业大学（深圳）计算机科学系，中国）
SimCMF: A Simple Cross-modal Fine-tuning Strategy from Vision Foundation Models to Any Imaging Modality
Center for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港中文大学人工智能机器人中心）；Princeton University（普林斯顿大学）；The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
Genome-Factory: A Library for Tuning, Deploying, and Interpreting Genomic Foundation Models
Center for Foundation Models；Generative AI \& Department of Computer Science, Northwestern University, USA；School of Natural Sciences, University of California at Merced, USA；Department of Energy Joint Genome Institute, Lawrence Berkeley National Laboratory, USA；Systems Biology Division, Lawrence Berkeley National Laboratory, USA；Department of Statistics；Data Science, Northwestern University, USA
Robustifying pathology foundation models via fine-tuning
Waiv；Institute of Pathology, Technical University of Munich（慕尼黑工业大学病理学研究所）；School of Computation, Information and Technology, Technical University of Munich（慕尼黑工业大学计算、信息与技术学院）
