The rapid development of large language models (LLMs) has been witnessed in recent years. Based on the powerful LLMs, multi-modal LLMs (MLLMs) extend the modality from text to a broader spectrum of domains, attracting widespread attention due to the broader range of application scenarios. As LLMs and MLLMs rely on vast amounts of model parameters and data to achieve emergent capabilities, the importance of data is receiving increasingly widespread attention and recognition. Tracing and analyzing recent data-oriented works for MLLMs, we find that the development of models and data is not two separate paths but rather interconnected. On the one hand, vaster and higher-quality data contribute to better performance of MLLMs; on the other hand, MLLMs can facilitate the development of data. The co-development of multi-modal data and MLLMs requires a clear view of 1) at which development stages of MLLMs specific data-centric approaches can be employed to enhance certain MLLM capabilities, and 2) how MLLMs, utilizing those capabilities, can contribute to multi-modal data in specific roles. To promote the data-model co-development for MLLM community, we systematically review existing works related to MLLMs from the data-model co-development perspective. A regularly maintained project associated with this survey is accessible at https://github.com/modelscope/data-juicer/blob/main/docs/awesome_llm_data.md.
Unveiling the Fragility of Vision-Language Models: Multi-Modal Adversarial Synergy via Texture-Constrained Perturbations and Cross-Modal Optimization
School of Software Engineering, Huazhong University of Science and Technology（华中科技大学软件学院）；Nanyang Technological University, Singapore（新加坡南洋理工大学）；University College London（伦敦大学学院）
ArgusCogito: Chain-of-Thought for Cross-Modal Synergy and Omnidirectional Reasoning in Camouflaged Object Segmentation
University of Chinese Academy of Sciences（中国科学院大学）；Technology and Engineering Center for Space Utilization, Chinese Academy of Sciences（中国科学院空间利用技术与工程中心）
Toward Knowledge-Guided AI for Inverse Design in Manufacturing: A Perspective on Domain, Physics, and Human-AI Synergy
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；KAIST InnoCORE PRISM-AI Center, Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院 InnoCORE PRISM-AI 中心）
Dependency, Compression, and Synergy: A Unified Information-Theoretic View of Multimodal Learning
School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics（西南财经大学计算机与人工智能学院）；Beijing Humanoid Robot Innovation Center (X-Humanoid)（北京人形机器人创新中心（X-Humanoid））；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Electronic Science and Technology of China（电子科技大学）
From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
School of Computer Science, Sichuan University（四川大学计算机学院）；School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School（北京大学深圳研究生院电子与计算机工程学院）；Institute of Artificial Intelligence (TeleAI), China Telecom and Northwestern Polytechnical University（中国电信与西北工业大学人工智能研究院（TeleAI））
How Vision Becomes Language: A Layer-wise Information-Theoretic Analysis of Multimodal Reasoning
Duke kunshan University, Duke University（杜克昆山大学、杜克大学）；The Chinese University of Hong Kong, Hong Kong, China（香港中文大学）；Fudan University, Shanghai, China（复旦大学）
SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation
School of Artificial Intelligence, Beihang University, Beijing, China（北京航空航天大学人工智能学院）；School of Computer Science and Engineering, Beihang University, Beijing, China（北京航空航天大学计算机科学与工程学院）
COFAP: A Universal Framework for COFs Adsorption Prediction through Designed Multi-Modal Extraction and Cross-Modal Synergy
College of Science, College of Information and Electrical Engineering（科学学院，信息与电气工程学院）；China Agricultural University（中国农业大学）；Qingdao Institute of Software, College of Computer Science and Technology（软件研究所，计算机科学与技术学院）；China University of Petroleum (East China)（中国石油大学（华东））；Weifang university（潍坊大学）；College of Environmental Science and Engineering（环境科学与工程学院）；North China Electric Power University（华北电力大学）；College of Science（科学学院）
A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；Innovation Center of Yangtze River Delta, Zhejiang University（浙江大学长三角创新中心）
More Perspectives, Stronger Signals: Multi-Perspective Enhancement and Progressive Fusion for Multimodal Entity Representation Learning
School of Computer Science, Hubei University（湖北大学计算机学院）；School of Cyber Science and Technology, Hubei University（湖北大学网络空间安全学院）；Institute of Vocational Education, Guangdong Industry Polytechnic University（广东轻工职业技术学院职业教育学院）；School of Computer Science, Hubei Key Laboratory of Big Data Intelligent Analysis and Application, Hubei University（湖北大学计算机学院、湖北省大数据智能分析与应用重点实验室）
Can Conversational Temporal Dynamics Improve Depression Detection in Dyads? A Preliminary Investigation in Multi-Modality Perspectives
Thomas Lord Department of Computer Science, University of Southern California, USA（汤姆斯·劳德计算机科学系，南加州大学，美国）；Signal Analysis and Interpretation Laboratory (SAIL), University of Southern California, USA（信号分析与解释实验室（SAIL），南加州大学，美国）
Graph Counselor: Adaptive Graph Exploration via Multi-Agent Synergy to Enhance LLM Reasoning
Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；School of Mathematics, Harbin Institute of Technology（哈尔滨工业大学数学学院）；Department of Control Science and Engineering, Harbin Institute of Technology（哈尔滨工业大学控制科学与工程学院）
