The quality of supervised fine-tuning (SFT) data is crucial for the performance of large multimodal models (LMMs), yet current data enhancement methods often suffer from factual errors and hallucinations due to inadequate visual perception. To address this challenge, we propose VERITAS, a pipeline that systematically integrates vision priors and multiple state-of-the-art LMMs with statistical methods to enhance SFT data quality. VERITAS leverages visual recognition models (RAM++) and OCR systems (PP-OCRv4) to extract structured vision priors, which are combined with images, questions, and answers. Three LMMs (GPT-4o, Gemini-2.5-Pro, Doubao-1.5-pro) evaluate the original answers, providing critique rationales and scores that are statistically fused into a high-confidence consensus score serving as ground truth. Using this consensus, we train a lightweight critic model via Group Relative Policy Optimization (GRPO), enhancing reasoning capabilities efficiently. Each LMM then refines the original answers based on the critiques, generating new candidate answers; we select the highest-scoring one as the final refined answer. Experiments across six multimodal benchmarks demonstrate that models fine-tuned with data processed by VERITAS consistently outperform those using raw data, particularly in text-rich and fine-grained reasoning tasks. Our critic model exhibits enhanced capability comparable to state-of-the-art LMMs while being significantly more efficient. We release our pipeline, datasets, and model checkpoints to advance research in multimodal data optimization.
GAMED: Knowledge Adaptive Multi-Experts Decoupling for Multimodal Fake News Detection
University of Southampton(南安普顿大学);University of Essex(埃塞克斯大学);Mohamed bin Zayed University of Artificial Intelligence(穆罕默德·本·扎耶德人工智能大学);The University of New South Wales(新南威尔士大学)
Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognition
School of Artificial Intelligence, Beijing University of Posts and Telecommunications(人工智能学院,北京邮电大学);School of Computer Science and Technology, Harbin Institute of Technology(计算机科学与技术学院,哈尔滨工业大学)
MGDT: MLLM-Guided Diffusion Transformer with Relation-Adaptive Mixture-of-Experts for Multimodal Knowledge Graph Completion
Beijing University of Posts and Telecommunications(北京邮电大学);Zhejiang University(浙江大学);University of Science and Technology of China(中国科学技术大学);Communication University of China(中国传媒大学)
Exploring Multimodal Foundation AI and Expert-in-the-Loop for Sustainable Management of Wild Salmon Fisheries in Indigenous Rivers
Simon Fraser University(西蒙弗雷泽大学);McGill University(麦吉尔大学);The University of Hong Kong(香港大学);Wild Salmon Center(野生鲑鱼中心);Pacific Salmon Foundation(太平洋鲑鱼基金会);Haida Fisheries Program(海达渔业计划)
CoRe-MoE: Compact Reusable MoE for Continual Multimodal Instruction Tuning
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络空间安全学院)
MMCTOP: A Multimodal Textualization and Mixture-of-Experts Framework for Clinical Trial Outcome Prediction
Nova School of Business and Economics(诺瓦商学院);Hogarthian Technologies(霍加斯技术);School of Medicine(医学院);Oregon Health & Science University(俄勒冈健康与科学大学);Cleveland Clinic(克利夫兰诊所);IBM Research(IBM研究院)
ProMoE-FL: Prototype-conditioned Mixture of Experts for Multimodal Federated Learning with Missing Modalities
NepAl Applied Mathematics and Informatics Institute(尼泊尔应用数学与信息学研究所);Fogsphere (Redev.AI Ltd)(福格球(Redev.AI有限公司));University of Lausanne(洛桑大学);West Virginia University(西弗吉尼亚大学);University of Verona(维罗纳大学);University College London(伦敦大学学院);University of Aberdeen(阿伯丁大学)
Beyond Spurious Signals: Debiasing Multimodal Large Language Models via Counterfactual Inference and Adaptive Expert Routing
School of Computer Science, Peking University(北京大学计算机科学学院);MOE Key Laboratory of Computational Linguistics, Peking University(北京大学教育部计算语言学重点实验室);National Key Laboratory for Multimedia Information Processing, Peking University(北京大学国家多媒体信息处理重点实验室)
MODE: Modality-Decomposed Expert-Level Mixed-Precision Quantization for MoE Multimodal LLMs
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);School of Artificial Intelligence, University of Chinese Academy of Sciences(中国科学院大学人工智能学院);Zhongguancun Academy(中关村学院)
Generalist-Specialist Mixture-of-Experts for Rare Pathology Detection in Multimodal Imaging
Technical University of Munich (TUM)(慕尼黑工业大学);TUM University Hospital(慕尼黑工业大学附属医院);Hasso Plattner Institute for Digital Engineering(哈索·普拉特纳数字工程研究所);University of Potsdam(波茨坦大学);Imperial College London(伦敦帝国理工学院)
ME-Mamba: Multi-Expert Mamba with Efficient Knowledge Capture and Fusion for Multimodal Survival Analysis
Digital Medical Research Center, School of Basic Medical Science, Fudan University, Shanghai 200032, China(复旦大学基础医学学院数字医学研究中心);Shanghai Key Lab of Medical Image Computing(上海医学图像计算重点实验室)
MoPE: Mixture of Prompt Experts for Parameter-Efficient and Scalable Multimodal Fusion
Department of Computing, The Hong Kong Polytechnic University(计算系,香港理工大学);Research Institute of Multiple Agents and Embodied Intelligence, Pengcheng Laboratory(多智能体与具身智能研究院,鹏城实验室)
Multi-level Mixture of Experts for Multimodal Entity Linking
School of Computer;Information Technology Shanxi University Taiyuan China;School of Computer Science;Informatics Cardiff University Cardiff UK;ILCC, School of Informatics University of Edinburgh Edinburgh UK;Information Technology Shanxi University;Informatics Cardiff University;ILCC, School of Informatics University of Edinburgh
CausalMoE: A Billion-Scale Multimodal Foundation Model for Granger Causal Discovery with Pattern-Routed Heterogeneous Experts
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University(北京大学智能科学与技术学院通用人工智能国家重点实验室);National Institute of Health Data Science, and Institute for Artificial Intelligence, Peking University(北京大学健康医疗大数据国家研究院、人工智能研究院)