In recent years, multimodal benchmarks for general domains have guided the rapid development of multimodal models on general tasks. However, the financial field has its peculiarities. It features unique graphical images (e.g., candlestick charts, technical indicator charts) and possesses a wealth of specialized financial knowledge (e.g., futures, turnover rate). Therefore, benchmarks from general fields often fail to measure the performance of multimodal models in the financial domain, and thus cannot effectively guide the rapid development of large financial models. To promote the development of large financial multimodal models, we propose MME-Finance, an bilingual open-ended and practical usage-oriented Visual Question Answering (VQA) benchmark. The characteristics of our benchmark are finance and expertise, which include constructing charts that reflect the actual usage needs of users (e.g., computer screenshots and mobile photography), creating questions according to the preferences in financial domain inquiries, and annotating questions by experts with 10+ years of experience in the financial industry. Additionally, we have developed a custom-designed financial evaluation system in which visual information is first introduced in the multi-modal evaluation process. Extensive experimental evaluations of 19 mainstream MLLMs are conducted to test their perception, reasoning, and cognition capabilities. The results indicate that models performing well on general benchmarks cannot do well on MME-Finance; for instance, the top-performing open-source and closed-source models obtain 65.69 (Qwen2VL-72B) and 63.18 (GPT-4o), respectively. Their performance is particularly poor in categories most relevant to finance, such as candlestick charts and technical indicator charts. In addition, we propose a Chinese version, which helps compare performance of MLLMs under a Chinese context.
MMedExpert-R1: Strengthening Multimodal Medical Reasoning via Domain-Specific Adaptation and Clinical Guideline Reinforcement
College of Computer Science and Software Engineering, Shenzhen University（深圳大学计算机科学与软件工程学院）；School of Artificial Intelligence, Shenzhen University（深圳大学人工智能学院）；Guangdong Provincial Key Laboratory of Intelligent Information Processing（广东省智能信息处理重点实验室）；The Hong Kong University of Science and Technology（香港科学与技术大学）；Renmin University of China（中国人民大学）；School of Biomedical Engineering, Shenzhen University（深圳大学生物医学工程学院）
MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models
The Chinese University of Hong Kong（香港中文大学）；Tongyi Lab（通义实验室）；SZTU（深圳技术大学）；Peking University（北京大学）；Tongji University（同济大学）；CASIA（中国科学院自动化所）；Tencent（腾讯）；UCLA（加州大学洛杉矶分校）；Westlake University（西湖大学）；NTU（国立大学）
MME-Survey: A Comprehensive Survey on Evaluation of Multimodal LLMs
Nanjing University（南京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Science and Technology of China（中国科学技术大学）；Nanyang Technological University（南洋理工大学）
MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs
Fudan University（复旦大学）；MMLab, The Chinese University of Hong Kong（中大香港人工智能实验室）；Shanghai AI Laboratory（上海人工智能实验室）；University of Science and Technology of China（中国科学技术大学）；Nanjing University（南京大学）
MegaPairs: Massive Data Synthesis For Universal Multimodal Retrieval
Beijing University of Posts and Telecommunications（北京邮电大学）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）；The Hong Kong Polytechnic University（香港理工大学）；University of Science and Technology of China（中国科学技术大学）；Shanghai Jiaotong University（上海交通大学）
Reasoning Guided Embeddings: Leveraging MLLM Reasoning for Improved Multimodal Retrieval
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；Sensetime Research（商汤科技研究院）；Beijing Institute of Technology（北京理工大学）；Shanghai AI Lab（上海人工智能实验室）
MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models
State Key Laboratory for Novel Software Technology, Nanjing University（南京大学新型软件技术国家重点实验室）；School of Intelligence Science and Technology, Nanjing University（南京大学智能科学与技术学院）；Tencent Youtu Lab（腾讯优图实验室）；Xiamen University（厦门大学）；CASIA（中国科学院自动化研究所）
Dynamic Multimodal Evaluation with Flexible Complexity by Vision-Language Bootstrapping
School of Artificial Intelligence, Shanghai Jiao Tong University（上海交通大学人工智能学院）；Shanghai AI Laboratory（上海人工智能实验室）；Tongji University（同济大学）；The University of Hong Kong（香港大学）
Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding
Shenzhen Technology University（深圳技术大学）；Shenzhen University of Information Technology（深圳信息科技学院）；University of Washington（华盛顿大学）；Foundation Model Team, Meituan（美团基础模型团队）；National University of Singapore（新加坡国立大学）；Tongji University（同济大学）
Mitigating Modality Prior-Induced Hallucinations in Multimodal Large Language Models via Deciphering Attention Causality
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；The Hong Kong University of Science and Technology（香港科技大学）；Nanyang Technological University（南洋理工大学）；Tsinghua University（清华大学）
Compressing then Matching: An Efficient Pre-training Paradigm for Multimodal Embedding
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences（人工智能安全国家重点实验室，计算技术研究所，中国科学院）；University of Chinese Academy of Sciences（中国科学院大学）；Kuaishou Technology（快手科技）
ReDiS: Reasoning-Diagnosis Self-Evolution for Multimodal Reasoning in Remote Sensing
School of Computer Science, Wuhan University（武汉大学计算机学院）；Zhongguancun Academy（中关村科学院）；National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing, Beijing Institute of Technology（北京理工大学天基智能信息处理科学与技术全国重点实验室）
