InfMAE: A Foundation Model in the Infrared Modality
School of Communications and Information Engineering, Chongqing University of Posts and Telecommunications（重庆邮电大学通信与信息工程学院）；School of Intelligent Systems Engineering, Sun Yat-sen University（中山大学智能系统工程学院）；School of Mathematics and Statistics, Xi’an Jiaotong University（西安交通大学数学与统计学院）；School of Computer and Information Technology, Shanxi University（山西大学计算机与信息技术学院）
In recent years, the foundation models have swept the computer vision field and facilitated the development of various tasks within different modalities. However, it remains an open question on how to design an infrared foundation model. In this paper, we propose InfMAE, a foundation model in infrared modality. We release an infrared dataset, called Inf30 to address the problem of lacking large-scale data for self-supervised learning in the infrared vision community. Besides, we design an information-aware masking strategy, which is suitable for infrared images. This masking strategy allows for a greater emphasis on the regions with richer information in infrared images during the self-supervised learning process, which is conducive to learning the generalized representation. In addition, we adopt a multi-scale encoder to enhance the performance of the pre-trained encoders in downstream tasks. Finally, based on the fact that infrared images do not have a lot of details and texture information, we design an infrared decoder module, which further improves the performance of downstream tasks. Extensive experiments show that our proposed method InfMAE outperforms other supervised methods and self-supervised learning methods in three downstream tasks.
SimMAT: Exploring Transferability from Vision Foundation Models to Any Image Modality
Center for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港创新研究院人工智能与机器人中心）；Princeton University（普林斯顿大学）；The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；Nanyang Technological University（南洋理工大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
SimCMF: A Simple Cross-modal Fine-tuning Strategy from Vision Foundation Models to Any Imaging Modality
Center for Artificial Intelligence and Robotics, HKISI, CAS（中国科学院香港中文大学人工智能机器人中心）；Princeton University（普林斯顿大学）；The Hong Kong Polytechnic University（香港理工大学）；The Hong Kong University of Science and Technology（香港科技大学）；State Key Laboratory of Multimodal Artificial Intelligence Systems, CASIA（中国科学院自动化研究所多模态人工智能系统国家重点实验室）
A Foundation Model for Brain Lesion Segmentation with Mixture of Modality Experts
Imperial College London（帝国理工学院）；UK Dementia Research Institute（英国痴呆症研究所）；Beijing Institute of Technology（北京理工大学）；Capital Medical University（首都医科大学）；Beijing Tiantan Hospital（北京天坛医院）
GeoMeld: Toward Semantically Grounded Foundation Models for Remote Sensing
Indian Institute of Technology Bombay（印度理工学院孟买分校）；Space Applications Centre, ISRO（印度空间研究组织空间应用中心）；University of Trento（特伦托大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
M-IDoL: Information Decomposition for Modality-Specific and Diverse Representation Learning in Medical Foundation Model
School of Computer Science and Technique（计算机科学与技术学院）；Tongji University（同济大学）；School of Communications and Electronic Engineering（通讯与电子工程学院）；East China Normal University（华东师范大学）
OpenMEDLab: An Open-source Platform for Multi-modality Foundation Models in Medicine
Shanghai AI Laboratory（上海人工智能实验室）；Shanghai Jiao Tong University（上海交通大学）；University of Electronic Science and Technology of China（电子科技大学）；Xi’an Jiaotong University（西安交通大学）；Zhejiang Laboratory（浙江实验室）；Fudan University（复旦大学）；Chinese University of Hong Kong（香港中文大学）；University of British Columbia（不列颠哥伦比亚大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；East China University Of Science And Technology（华东理工大学）；University College London（伦敦大学学院）；Guangzhou Laboratory（广州实验室）；The First Affiliated Hospital of Zhengzhou University（郑州大学第一附属医院）；West China Hospital, Sichuan University（四川大学华西医院）；Xinhua Hospital, Shanghai Jiaotong University School of Medicine（上海交通大学医学院附属新华医院）；Ruijin Hospital, Shanghai Jiaotong University School of Medicine（上海交通大学医学院附属瑞金医院）
RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation
Aerospace Information Research Institute, Chinese Academy of Sciences（中国科学院航天信息研究所）；School of Electronic, Electrical and Communication Engineering, University of Chinese Academy of Sciences（中国科学院大学电子电气与通信工程学院）；University of Chinese Academy of Sciences（中国科学院大学）；Key Laboratory of Target Cognition and Application Technology(TCAT), Aerospace Information Research Institute, Chinese Academy of Sciences（目标认知与应用技术重点实验室）；Department of Electronic Engineering, Tsinghua University（清华大学电子工程系）；Peng Cheng Laboratory, Shenzhen（鹏城实验室）
VGGSounder: Audio-Visual Evaluations for Foundation Models
Technical University of Munich, MCML（慕尼黑技术大学，MCML）；University of Tübingen（图宾根大学）；Tübingen AI Center（图宾根人工智能中心）；MPI for Intelligent Systems, ELLIS Institute（智能系统Max Planck研究所，ELLIS研究所）
Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration
DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Hupan Lab（湖畔实验室）；College of Computer Science and Technology, Zhejiang University, China（浙江大学计算机科学与技术学院）；Shengjing Hospital of China Medical University, China（中国医科大学附属盛京医院）
(Almost) Free Modality Stitching of Foundation Models
University of Tübingen（图宾根大学）；Zuse School ELIZA（Zuse学校ELIZA）；ELLIS Institute Tübingen（图宾根ELLIS研究所）；MPI-IS Tübingen（图宾根MPI-IS研究所）；SAIT AI Lab Montréal（蒙特利尔SAIT人工智能实验室）；Tübingen AI Center（图宾根人工智能中心）
Mind the Modality Gap: Towards a Remote Sensing Vision-Language Model via Cross-modal Alignment
organization= Orion Lab, National Observatory of Athens \& National Technical University of Athens , country= Greece；organization= Department of Informatics \& Telematics, Harokopio University of Athens , country= Greece；organization= Faculty of Electrical Engineering；organization= BIFOLD - Berlin Institute for the Foundations of Learning
