Math-LLaVA: Bootstrapping Mathematical Reasoning for Multimodal Large Language Models
University of Electronic Science and Technology of China(电子科技大学);Tongji University(同济大学);National University of Singapore(新加坡国立大学);Singapore University of Technology and Design(新加坡科技设计大学)
Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang, See-Kiong Ng, Lidong Bing, Roy Ka-Wei Lee
Large language models (LLMs) have demonstrated impressive reasoning capabilities, particularly in textual mathematical problem-solving. However, existing open-source image instruction fine-tuning datasets, containing limited question-answer pairs per image, do not fully exploit visual information to enhance the multimodal mathematical reasoning capabilities of Multimodal LLMs (MLLMs). To bridge this gap, we address the lack of high-quality, diverse multimodal mathematical datasets by collecting 40K high-quality images with question-answer pairs from 24 existing datasets and synthesizing 320K new pairs, creating the MathV360K dataset, which enhances both the breadth and depth of multimodal mathematical questions. We introduce Math-LLaVA, a LLaVA-1.5-based model fine-tuned with MathV360K. This novel approach significantly improves the multimodal mathematical reasoning capabilities of LLaVA-1.5, achieving a 19-point increase and comparable performance to GPT-4V on MathVista's minitest split, and yielding leading performance on Math-V and MathVerse. Furthermore, Math-LLaVA demonstrates enhanced generalizability, showing substantial improvements on the MMMU benchmark. Our research highlights the importance of dataset diversity and synthesis in advancing MLLMs' mathematical reasoning abilities. The code and data are available at: \url{https://github.com/HZQ950419/Math-LLaVA}.
Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution
Alibaba Group Holding Limited(阿里巴巴集团控股有限公司);EPIC Lab, Shanghai Jiao Tong University(上海交通大学EPIC实验室);Shanghai University of Finance and Economics(上海财经大学);Wuhan University(武汉大学);Zhejiang University(浙江大学)
National Key Laboratory for Novel Software Technology, Nanjing University(南京大学计算机软件新技术国家重点实验室);China Mobile Communications Company Limited Research Institute(中国移动通信有限公司研究院)
$\mathcal{V}isi\mathcal{P}runer$: Decoding Discontinuous Cross-Modal Dynamics for Efficient Multimodal LLMs
Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative, Institute of Digital Twin, EIT, Ningbo(宁波空间智能与数字衍生关键实验室,数字孪生研究院,EIT,宁波);Shanghai Jiao Tong University(上海交通大学);Hong Kong Polytechnic University(香港理工大学);Meituan Inc.(美团公司);National University of Singapore(新加坡国立大学)
MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models
University of Cambridge(剑桥大学);Southern University of Science and Technology(南方科技大学);Hong Kong University of Science and Technology(香港科技大学);Huawei Noah’s Ark Lab(华为诺亚方舟实验室);The Alan Turing Institute(艾伦·图灵研究所);Max Planck Institute for Intelligent Systems - Tübingen(马克斯·普朗克智能系统研究所-蒂宾根)
VisualWebInstruct: Scaling up Multimodal Instruction Data through Web Search
University of Waterloo(滑铁卢大学);University of Toronto(多伦多大学);University of California, Santa Barbara(加利福尼亚大学圣巴巴拉分校);Carnegie Mellon University(卡内基梅隆大学);Nanyang Technological University(南洋理工大学)