arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2408.01319cs.AI
收藏

多模态大语言模型综合综述:跨不同任务的性能与挑战

A Comprehensive Review of Multimodal Large Language Models: Performance and Challenges Across Different Tasks

School of Computer Science, Northwestern Polytechnical University(西北工业大学计算机学院);School of Automation, Northwestern Polytechnical University(西北工业大学自动化学院);Institute of Medical Research, Northwestern Polytechnical University(西北工业大学医学研究院);School of Computing, The University of Georgia(佐治亚大学计算学院);School of Physics and Information Technology, Shaanxi Normal University(陕西师范大学物理与信息技术学院)
Jiaqi Wang, Hanqi Jiang, Yiheng Liu, Chong Ma, Xu Zhang, Yi Pan, Mengyuan Liu, Peiran Gu, Sichen Xia, Lifeng Chen, Junhao Chen, Wenjun Li, Yutong Zhang, Zihao W… 展开作者

Jiaqi Wang, Hanqi Jiang, Yiheng Liu, Chong Ma, Xu Zhang, Yi Pan, Mengyuan Liu, Peiran Gu, Sichen Xia, Lifeng Chen, Junhao Chen, Wenjun Li, Yutong Zhang, Zihao Wu, Zhengliang Liu, Tianyang Zhong, Bao Ge, Tuo Zhang, Ning Qiang, Xintao Hu, Xi Jiang, Xin Zhang, Wei Zhang, Dinggang Shen, Tianming Liu, Shu Zhang

AI总结:

本文系统综述了多模态大语言模型在自然语言、视觉和音频等任务中的应用,比较了不同模型的侧重点,分析了当前不足并展望未来研究方向。

摘要:

在数据爆炸性增长和技术快速进步的时代,多模态大语言模型(MLLMs)处于人工智能(AI)系统的最前沿。MLLMs旨在无缝整合包括文本、图像、视频、音频和生理序列在内的多种数据类型,以应对远超单模态系统能力的真实世界应用复杂性。本文系统梳理了MLLM在自然语言、视觉和音频等多模态任务中的应用。我们还对不同MLLM在任务中的侧重点进行了比较分析,并对当前MLLM的不足提出了见解,同时指出了未来研究的潜在方向。通过这些讨论,本文希望为MLLM的进一步发展和应用提供有价值的见解。

英文摘要:

In an era defined by the explosive growth of data and rapid technological advancements, Multimodal Large Language Models (MLLMs) stand at the forefront of artificial intelligence (AI) systems. Designed to seamlessly integrate diverse data types-including text, images, videos, audio, and physiological sequences-MLLMs address the complexities of real-world applications far beyond the capabilities of single-modality systems. In this paper, we systematically sort out the applications of MLLM in multimodal tasks such as natural language, vision, and audio. We also provide a comparative analysis of the focus of different MLLMs in the tasks, and provide insights into the shortcomings of current MLLMs, and suggest potential directions for future research. Through these discussions, this paper hopes to provide valuable insights for the further development and application of MLLM.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑