Unified Molecule Pre-training with Flexible 2D and 3D Modalities: Single and Paired Modality Integration
Computational Bioscience Research Center, King Abdullah University of Science and Technology(国王阿卜杜勒·阿齐兹科技大学计算生物科学研究中心);Institute for Infocomm Research, A*STAR(信息与通信研究机构,A*STAR);School of Computing and Information Systems, Singapore Management University(新加坡管理大学计算机与信息系统学院)
Molecular representation learning plays a crucial role in advancing applications such as drug discovery and material design. Existing work leverages 2D and 3D modalities of molecular information for pre-training, aiming to capture comprehensive structural and geometric insights. However, these methods require paired 2D and 3D molecular data to train the model effectively and prevent it from collapsing into a single modality, posing limitations in scenarios where a certain modality is unavailable or computationally expensive to generate. To overcome this limitation, we propose FlexMol, a flexible molecule pre-training framework that learns unified molecular representations while supporting single-modality input. Specifically, inspired by the unified structure in vision-language models, our approach employs separate models for 2D and 3D molecular data, leverages parameter sharing to improve computational efficiency, and utilizes a decoder to generate features for the missing modality. This enables a multistage continuous learning process where both modalities contribute collaboratively during training, while ensuring robustness when only one modality is available during inference. Extensive experiments demonstrate that FlexMol achieves superior performance across a wide range of molecular property prediction tasks, and we also empirically demonstrate its effectiveness with incomplete data. Our code and data are available at https://github.com/tewiSong/FlexMol.
Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs
School of Information Science and Engineering, Yunnan University(云南大学信息科学与工程学院);School of Software, Yunnan University(云南大学软件学院);National University of Singapore(新加坡国立大学);School of Engineering, Yunnan University(云南大学工程学院)
Token-Level Contrastive Learning with Modality-Aware Prompting for Multimodal Intent Recognition
Tsinghua University(清华大学);Beijing National Research Center for Information Science and Technology(北京信息科学与技术国家研究中心);Hebei University of Science and Technology(河北科技大学)
OVA-IB: One vs All Information Bottleneck for Multi-Modal Alignment
Hong Kong University of Science and Technology(香港科学与技术大学);Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);UiT – The Arctic University of Norway(挪威北极大学);University of Copenhagen(哥本哈根大学);Norwegian Computing Center(挪威计算中心);University of Massachusetts Amherst(马萨诸塞大学阿姆赫斯特分校)
School of Computer Science and Engineering, Southeast University, Nanjing, China(计算机科学与工程学院,东南大学,南京,中国);Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China(新一代人工智能技术及其交叉应用重点实验室(东南大学),教育部,中国)
Addressing Missing and Noisy Modalities in One Solution: Unified Modality-Quality Framework for Low-quality Multimodal Data
Department of Computer Science(计算机科学系);South China Normal University(华南师范大学);School of Electronics and Information Technology(电子与信息学院);Sun Yat-sen University(中山大学)
Multi-modal Co-learning for Earth Observation: Enhancing single-modality models via modality collaboration
Department of Computer Science, University of Kaiserslautern-Landau (RPTU)(科斯拉尔特伦大学计算机科学系);SDS, German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心(DFKI));INRAE, UMR TETIS, University of Montpellier(蒙彼利埃大学UMR TETIS);CIRAD, UMR TETIS, University of Montpellier(蒙彼利埃大学UMR TETIS);INRIA, EVERGREEN, University of Montpellier(蒙彼利埃大学)
Enhancing multimodal cooperation via sample-level modality valuation
Gaoling School of Artificial Intelligence, Renmin University of China(中国人民大学高瓴人工智能学院);Beijing Key Laboratory of Big Data Management and Analysis Methods(北京大数据管理与分析方法重点实验室)
Sydney AI Centre, The University of Sydney(悉尼人工智能中心,悉尼大学);Hong Kong Baptist University(香港 Baptist 大学);RIKEN Center for Advanced Intelligence Project(理化学研究所先进智能项目中心);The University of Tokyo(东京大学)
Co-Learning for Missing Arbitrary Modalities in Multi-modal Classification
GFZ Helmholtz Center for Geosciences(德国波茨坦地学研究中心亥姆霍兹中心);INRAE, UMR TETIS, University of Montpellier(法国蒙彼利埃大学农业环境与生态研究院、蒙彼利埃大学TETIS联合研究单位);INRIA, EVERGREEN, University of Montpellier(法国蒙彼利埃大学信息与自动化研究所、EVERGREEN实验室);CIRAD, UMR TETIS, University of Montpellier(法国蒙彼利埃大学国际农业研究磋商组织、蒙彼利埃大学TETIS联合研究单位)
Cross-Modal Prototype based Multimodal Federated Learning under Severely Missing Modality
Kyung Hee University(庆熙大学);The University of Danang—Vietnam-Korea University of Information and Communication Technology(岘港大学—越韩信息与通信技术大学);Digital Science and Technology Institute(数字科学技术研究院)
Multimodal Sentiment Analysis with Missing Modality: A Knowledge-Transfer Approach
College of Computing and Data Science, Nanyang Technological University, Singapore(南洋理工大学计算机与数据科学学院);Singapore University of Social Sciences, Singapore(新加坡社会科学研究大学)