Connecting text and visual modalities plays an essential role in generative intelligence. For this reason, inspired by the success of large language models, significant research efforts are being devoted to the development of Multimodal Large Language Models (MLLMs). These models can seamlessly integrate visual and textual modalities, while providing a dialogue-based interface and instruction-following capabilities. In this paper, we provide a comprehensive review of recent visual-based MLLMs, analyzing their architectural choices, multimodal alignment strategies, and training techniques. We also conduct a detailed analysis of these models across a wide range of tasks, including visual grounding, image generation and editing, visual understanding, and domain-specific applications. Additionally, we compile and describe training datasets and evaluation benchmarks, conducting comparisons among existing models in terms of performance and computational requirements. Overall, this survey offers a comprehensive overview of the current state of the art, laying the groundwork for future MLLMs.
AI-Driven HSI: Multimodality, Fusion, Challenges, and the Deep Learning Revolution
School of Electrical Engineering and Computer Science, Gwangju Institute of Science and Technology (GIST)（光州科学技术院电气工程与计算机科学学院）；Artificial Intelligence Graduate School, GIST（光州科学技术院人工智能研究生院）
A Survey of Multimodal Large Language Model from A Data-centric Perspective
Hong Kong University of Science and Technology（香港科技大学）；Apple（苹果公司）；Peking University（北京大学）；Harbin Institute of Technology（哈尔滨工业大学）；University of Science and Technology of China（中国科学技术大学）
LLMs Meet Multimodal Generation and Editing: A Survey
The Hong Kong University of Science and Technology（香港科技大学）；The Chinese University of Hong Kong（香港中文大学）；Tsinghua University（清华大学）；Tencent AI Lab（腾讯AI实验室）
How to Bridge the Gap between Modalities: Survey on Multimodal Large Language Model
College of Computer, National University of Defense Technology（国防科技大学计算机学院）；School of Computer and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）
From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning
University of Pittsburgh（匹兹堡大学）；Stanford University（斯坦福大学）；UCSD（加州大学圣地亚哥分校）；University of Arizona（亚利桑那大学）；Georgia Tech（佐治亚理工学院）；Carnegie Mellon University（卡内基梅隆大学）；UNC Chapel Hill（北卡罗来纳大学教堂山分校）；UC Merced（加州大学默塞德分校）；New York University（纽约大学）
From Structure to Synergy: A Survey of Vision-Language Perception Paradigm Evolution in Multimodal Large Language Models
School of Computer Science, Sichuan University（四川大学计算机学院）；School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School（北京大学深圳研究生院电子与计算机工程学院）；Institute of Artificial Intelligence (TeleAI), China Telecom and Northwestern Polytechnical University（中国电信与西北工业大学人工智能研究院（TeleAI））
Empowering Multimodal LLMs with External Tools: A Comprehensive Survey
School of Automation Science and Engineering, Xi’an Jiaotong University（自动化科学与工程学院，西安交通大学）；Interdisciplinary Graduate Programme, Nanyang Technological University（跨学科研究生项目，南洋理工大学）；Lenovo Research, Lenovo（联想研究院，联想）；School of Computer Science and Technology, Xi’an Jiaotong University（计算机科学与技术学院，西安交通大学）；College of Computing and Data Science, Nanyang Technological University（计算与数据科学学院，南洋理工大学）
MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs
Nanjing University（南京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Science and Technology of China（中国科学技术大学）；Nanyang Technological University（南洋理工大学）
A Survey of Generative Categories and Techniques in Multimodal Generative Models
School of Architecture, Technology and Engineering, University of Brighton, Lewes Road, BN2 4GJ（建筑、科技与工程学院，布里顿大学，勒斯路，BN2 4GJ）；University of Khorfakkan, Sharjah（柯法克坎大学，沙迦）
Ask in Any Modality: A Comprehensive Survey on Multimodal Retrieval-Augmented Generation
Computer Engineering Department, Sharif University of Technology（沙斐大学计算机工程系）；College of Interdisciplinary Science and Technology, University of Tehran（德黑兰大学跨学科科学与技术学院）；Computer Engineering Department, K.N. Toosi University of Technology（技术学院计算机工程系）；Qatar Computing Research Institute（卡塔尔计算研究院）
Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Survey
Peking University（北京大学）；Beihang Univerisy（北京航空航天大学）；University of Hongkong（香港大学）；Shenzhen Institute of Advanced Technology, China Academy of Sciences（中国科学院深圳先进技术研究院）；Tsinghua University（清华大学）；M-A-P；The Chinese University of Hong Kong（香港中文大学）；Alibaba Group（阿里巴巴集团）；University of Cambridge（剑桥大学）；Microsoft Research（微软研究院）；UIUC（伊利诺伊大学厄巴纳-香槟分校）；Humanify Inc.（Humanify公司）；Zhejiang University（浙江大学）
Personalized Multimodal Large Language Models: A Survey
University of California, San Diego（加利福尼亚大学圣地亚哥分校）；University of Rochester（罗切斯特大学）；Dartmouth College（达特茅斯学院）；Adobe Research（奥多比研究院）；University of Massachusetts at Amherst（马萨诸塞大学阿默斯特分校）；University of Southern California（南加利福尼亚大学）；Virginia Tech（弗吉尼亚理工大学）；Cisco Research（思科研究院）；University of Oregon（俄勒冈大学）；Meta AI；University of Arizona（亚利桑那大学）；University of Maryland（马里兰大学）
Multimodal Information Fusion for Chart Understanding: A Survey of MLLMs -- Evolution, Limitations, and Cognitive Enhancement
College of Computer Science, Sichuan University（四川大学计算机科学学院）；Engineering Research Center of Machine Learning（机器学习工程研究中心）；China Telecom Institute of AI（中国电信人工智能研究院）
A Survey of Multimodal Ophthalmic Diagnostics: From Task-Specific Approaches to Foundational Models
College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China（深圳大学计算机科学与软件工程学院）；Shenzhen Key Laboratory of Visual Object Detection and Recognition, Harbin Institute of Technology, Shenzhen, 518055, China（视觉对象检测与识别深圳重点实验室）；Laboratory for Artificial Intelligence in Design, Hong Kong（人工智能设计实验室）；School of Artificial Intelligence, Shenzhen University, Shenzhen, China（深圳大学人工智能学院）
Survey of Large Multimodal Model Datasets, Application Categories and Taxonomy
University of Washington（华盛顿大学）；New York University（纽约大学）；Columbia University（哥伦比亚大学）；Liverpool John Moores University（利物浦约翰摩尔大学）；Chennai Mathematical Institute（金奈数学研究所）；Birla Institute of Technology（比拉理工学院）
Foundations and Recent Trends in Multimodal Mobile Agents: A Survey
Australian Artificial Intelligence Institute（澳大利亚人工智能研究所）；The Pennsylvania State University（宾夕法尼亚州立大学）；Beijing Jiaotong University（北京交通大学）；University of Liverpool（利物浦大学）
A Survey of Mathematical Reasoning in the Era of Multimodal Large Language Model: Benchmark, Method & Challenges
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Nanyang Technological University（南洋理工大学）；Squirrel Ai Learning（松鼠Ai学习）
Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
HKUST(GZ)（香港科技大学（广州））；HKUST（香港科技大学）；South China University of Technology（华南理工大学）；Shanghai Jiao Tong University（上海交通大学）；University of Pisa（比萨大学）；University of Trento（特伦托大学）
Self-Improvement in Multimodal Large Language Models: A Survey
The University of Texas at Dallas（德克萨斯大学达拉斯分校）；University of Toronto（多伦多大学）；University of Notre Dame（诺特丹大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）
The Paradigm Shift: A Comprehensive Survey on Large Vision Language Models for Multimodal Fake News Detection
College of Computer and Mathematics, Central South University of Forestry and Technology（计算机与数学学院，中央南大学林业科技学院）；College of Computer Science and Electronic Engineering, Hunan University（计算机科学与电子工程学院，湖南大学）；College of Economics and Management, Central South University of Forestry and Technology（经济管理学院，中央南大学林业科技学院）；Department of Computer Science, State University of New York（计算机科学系，纽约州立大学）
Can Large Language Models Revolutionize Survey Research? Experiments with Disaster Preparedness Responses
Dept. of Urban and Regional Planning & Florida Institute for Built Environment Resilience, University of Florida（城市与区域规划系及佛罗里达环境韧性研究所，佛罗里达大学）；College of Liberal Arts and Sciences, Bureau of Economic and Business Research, University of Florida（文理学院及经济与商业研究局，佛罗里达大学）
From Efficient Multimodal Models to World Models: A Survey
Fudan University（复旦大学）；Academy for Engineering and Technology, Fudan University（复旦大学工程与技术学院）；Shanghai Engineering Research Center of AI and Robotics（上海人工智能与机器人工程研究中心）；Engineering Research Center of AI and Robotics, Ministry of Education（教育部人工智能与机器人工程研究中心）；Shanghai Key Lab of Intelligent Information Processing（上海智能信息处理重点实验室）；School of Computer Science, Fudan University（复旦大学计算机科学技术学院）
