Recent efforts have explored multimodal semantic segmentation using various backbone architectures. However, while most methods aim to improve accuracy, their computational efficiency remains underexplored. To address this, we propose EGFormer, an efficient multimodal semantic segmentation framework that flexibly integrates an arbitrary number of modalities while significantly reducing model parameters and inference time without sacrificing performance. Our framework introduces two novel modules. First, the Any-modal Scoring Module (ASM) assigns importance scores to each modality independently, enabling dynamic ranking based on their feature maps. Second, the Modal Dropping Module (MDM) filters out less informative modalities at each stage, selectively preserving and aggregating only the most valuable features. This design allows the model to leverage useful information from all available modalities while discarding redundancy, thus ensuring high segmentation quality. In addition to efficiency, we evaluate EGFormer on a synthetic-to-real transfer task to demonstrate its generalizability. Extensive experiments show that EGFormer achieves competitive performance with up to 88 percent reduction in parameters and 50 percent fewer GFLOPs. Under unsupervised domain adaptation settings, it further achieves state-of-the-art transfer performance compared to existing methods.
Multimodal Causal-Driven Representation Learning for Generalizable Medical Image Segmentation
City University of Hong Kong(香港城市大学);Shenzhen Loop Area Institute(深圳河套学院);CAIR, HKISI, Chinese Academy of Sciences(中国科学院计算智能研究所);UESTC(电子科技大学);MAIS, Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院)
Extracting Multimodal Learngene in CLIP: Unveiling the Multimodal Generalizable Knowledge
School of Computer Science and Engineering, Southeast University, Nanjing 210096, China(东南大学计算机科学与工程学院);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及其跨学科应用关键实验室)
ProMMSearchAgent: A Generalizable Multimodal Search Agent Trained with Process-Oriented Rewards
East China Normal University(东中国师范大学);Shanghai Innovation Institute(上海创新研究院);Huawei Noah's Ark Lab(华为诺亚实验室);Independent Researcher(独立研究者);University College London(伦敦大学学院)
GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Control
École Polytechnique Fédérale de Lausanne (EPFL)(洛桑联邦理工学院(EPFL));University of Bern(伯尔尼大学);Swiss Data Science Center(瑞士数据科学中心);University of Zurich(苏黎世大学);ETH Zurich(苏黎世联邦理工学院)
Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detection
School of Software Engineering, Xi’an Jiaotong University(西安交通大学软件工程学院);School of Computer Science and Information Engineering, Hefei University of Technology(合肥工业大学计算机科学与信息工程学院);CSIRO(澳大利亚联邦科学与工业研究组织);Northwestern Polytechnical University(西北工业大学);University of Macau(澳门大学)
MGPC: Multimodal Network for Generalizable Point Cloud Completion With Modality Dropout and Progressive Decoding
School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation of Chinese Academy of Sciences(中国科学院自动化研究所 multimodal 人工智能系统国家重点实验室);Chemical Defense Institute, Academy of Military Sciences(军事科学院化学防御研究院);Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation of Chinese Academy of Sciences(中国科学院自动化研究所 复杂系统认知与决策智能重点实验室)
MIFNet: Learning Modality-Invariant Features for Generalizable Multimodal Image Matching
National Engineering Research Center for Multimedia Software(多媒体软件国家工程研究中心);Institute of Artificial Intelligence(人工智能研究院);School of Computer Science(计算机科学学院);Hubei Key Laboratory of Multimedia and Network Communication Engineering(湖北省多媒体与网络通信工程重点实验室);Lee Kong Chian School of Medicine(李光耀医学院);Nanyang Technological University(南洋理工大学);Ningbo Institute of Materials Technology and Engineering(宁波材料技术与工程研究所);Chinese Academy of Sciences(中国科学院);Institute for Infocomm Research (I 2 R)(信息与通信研究所以(I 2 R));Agency for Science, Technology and Research (A*STAR)(科技研究局(A*STAR))
AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University(中国教育部多媒体可信感知与高效计算重点实验室,厦门大学);Tencent YouTu Lab(腾讯YouTu实验室)
A multimodal vision foundation model for generalizable knee pathology
Beihang University(北京航空航天大学);Peking University Third Hospital(北京大学第三医院);Institute of Sports Medicine of Peking University(北京大学运动医学研究所);Ministry of Education(教育部)
DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Action
MoE Key Laboratory of Brain-inspired Intelligent Perception and Cognition, University of Science and Technology of China(脑启发智能感知与认知国家重点实验室,中国科学技术大学);State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University(多媒体信息处理国家重点实验室,北京大学计算机学院);CUHK(香港大学)
Affordance-R1: Reinforcement Learning for Generalizable Affordance Reasoning in Multimodal Large Language Model
The Hong Kong University of Science and Technology (GZ)(香港科技大学(广州));National University of Singapore(新加坡国立大学);ShanghaiTech University(上海科技大学);East China Normal University(华东师范大学);Nanjing University of Information Science & Technology(南京信息工程大学);Zhejiang University(浙江大学);Institute of Automation, Chinese Academy of Science(中国科学院自动化研究所);Shanghai AI Laboratory(上海人工智能实验室)
FlightGPT: Towards Generalizable and Interpretable UAV Vision-and-Language Navigation with Vision-Language Models
School of Intelligent Systems Engineering, Sun Yat-Sen University(中山大学智能系统工程学院);DP Technology(DP技术公司);Beijing University Of Posts and Telecommunications(北京邮电大学);Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);Tongji University(同济大学);School of Integrated Innovation, Chulalongkorn University(朱拉隆功大学创新学院)
MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image
Technion – Israel Institute of Technology(技术ion – 以色列理工学院);Prior Labs(Prior实验室);NVIDIA;SODA Team, INRIA Saclay, Palaiseau(SODA团队,INRIA萨克莱,帕莱索);University of Freiburg(弗赖堡大学);Probabl;ELLIS Institute Tübingen(图宾根ELLIS研究所)
From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning
University of Pittsburgh(匹兹堡大学);Stanford University(斯坦福大学);UCSD(加州大学圣地亚哥分校);University of Arizona(亚利桑那大学);Georgia Tech(佐治亚理工学院);Carnegie Mellon University(卡内基梅隆大学);UNC Chapel Hill(北卡罗来纳大学教堂山分校);UC Merced(加州大学默塞德分校);New York University(纽约大学)