多模态大模型

LecEval: An Automated Metric for Multimodal Knowledge Acquisition in Multimedia Learning

Joy Lim Jia Yin, Daniel Zhang-Li, Jifan Yu, Haoxuan Li, Shangqing Tu, Yuanchun Wang, Zhiyuan Liu, Huiqin Liu, Lei Hou, Juanzi Li, Bin Xu

机构 * Tsinghua University（清华大学） ； Beihang University（北航） ； Renmin University of China（中国人民大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments 6 pages, 3 figures

Multimodal Large Language Models for Medicine: A Comprehensive Survey

Jiarui Ye, Hao Tang

机构 * School of Computer Science and Engineering, Nanjing University of Science and Technology（南京理工大学计算机科学与工程学院） ； School of Computer Science, Peking University（北京大学计算机科学学院） 纠错

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CL、cs.MM

mrCAD: Multimodal Refinement of Computer-aided Designs

William P. McCarthy, Saujas Vaduguru, Karl D. D. Willis, Justin Matejka, Judith E. Fan, Daniel Fried, Yewen Pu

机构 * Autodesk AI Lab（Autodesk AI实验室） ； Carnegie Mellon University（卡内基梅隆大学） ； Stanford University（斯坦福大学） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Comments the first two authors contributed equally

Memory Reviving, Continuing Learning and Beyond: Evaluation of Pre-trained Encoders and Decoders for Multimodal Machine Translation

Zhuang Yu, Shiliang Sun, Jing Zhao, Tengfei Song, Hao Yang

机构 * Department of Automation, Shanghai Jiao Tong University（上海交通大学自动化系） ； School of Computer Science and Technology, East China Normal University（华东师范大学计算机科学与技术学院） ； Labs, Huawei Technologies CO., LTD（华为技术有限公司2012实验室） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

M4U: Evaluating Multilingual Understanding and Reasoning for Large Multimodal Models

Hongyu Wang, Jiayu Xu, Senwei Xie, Ruiping Wang, Jialin Li, Zhaojie Xie, Bin Zhang, Chuyan Xiong, Xilin Chen

机构 * Key Laboratory of AI Safety of Chinese Academy of Sciences (CAS)（中国科学院人工智能安全重点实验室（中国科学院）） ； Institute of Computing Technology, CAS（中国科学院计算技术研究所） 纠错

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments Work in progress

Multimodal LLM Augmented Reasoning for Interpretable Visual Perception Analysis

Shravan Chaudhari, Trilokya Akula, Yoon Kim, Tom Blake

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Multimodal Fake News Video Explanation: Dataset, Analysis and Evaluation

Lizhi Chen, Zhong Qian, Peifeng Li, Qiaoming Zhu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.MM

Imagery as Inquiry: Exploring A Multimodal Dataset for Conversational Recommendation

Se-eun Yoon, Hyunsik Jeon, Julian McAuley

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

PuzzleBench: A Fully Dynamic Evaluation Framework for Large Multimodal Models on Puzzle Solving

Zeyu Zhang, Zijian Chen, Zicheng Zhang, Yuze Sun, Yuan Tian, Ziheng Jia, Chunyi Li, Xiaohong Liu, Xiongkuo Min, Guangtao Zhai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts

Jiansheng Li, Xingxuan Zhang, Hao Zou, Yige Guo, Renzhe Xu, Yilong Liu, Chuzhao Zhu, Yue He, Peng Cui

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios

Lu Qiu, Yi Chen, Yuying Ge, Yixiao Ge, Ying Shan, Xihui Liu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments Code & data are available at: https://qiulu66.github.io/egoplanbench2/

Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset

Shiao Wang, Xiao Wang, Bo Jiang, Lin Zhu, Guoqi Li, Yaowei Wang, Yonghong Tian, Jin Tang

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

Comments Journal Extension of HARDVS (AAAI 2024)

MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models

Pengfei Zhou, Fanrui Zhang, Xiaopeng Peng, Zhaopan Xu, Jiaxin Ai, Yansheng Qiu, Chuanhao Li, Zhen Li, Ming Li, Yukang Feng, Jianwen Sun, Haoquan Zhang, Zizhen Li, Xiaofeng Mao, Wangbo Zhao, Kai Wang, Xiaojun Chang, Wenqi Shao, Yang You, Kaipeng Zhang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 11 pages, 8 figures

Enhancing the Reasoning Ability of Multimodal Large Language Models via Mixed Preference Optimization

Weiyun Wang, Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, Jifeng Dai

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Diagnosis of Pulmonary Hypertension by Integrating Multimodal Data with a Hybrid Graph Convolutional and Transformer Network

Fubao Zhu, Yang Zhang, Gengmin Liang, Jiaofen Nan, Yanting Li, Chuang Han, Danyang Sun, Zhiguo Wang, Chen Zhao, Wenxuan Zhou, Jian He, Yi Xu, Iokfai Cheang, Xu Zhu, Yanli Zhou, Weihua Zhou

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments 23 pages, 8 figures, 4 tables

RECALL-MM: A Multimodal Dataset of Consumer Product Recalls for Risk Analysis using Computational Methods and Large Language Models

Diana Bolanos, Mohammadmehdi Ataei, Daniele Grandi, Kosa Goucher-Lambert

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

A large-scale image-text dataset benchmark for farmland segmentation

Chao Tao, Dandan Zhong, Weiliang Mu, Zhuofei Du, Haiyang Wu

专题命中 多模态评测 ：image-text(title,abstract)；分类 cs.CV、cs.CL

Can Multi-modal (reasoning) LLMs work as deepfake detectors?

Simiao Ren, Yao Yao, Kidus Zewde, Zisheng Liang, Tsang, Ng, Ning-Yau Cheng, Xiaoou Zhan, Qinzhe Liu, Yifei Chen, Hengwei Xu

专题命中 多模态评测 ：multi-modal(title,abstract)；分类 cs.CV、cs.AI

MMIE: Massive Multimodal Interleaved Comprehension Benchmark for Large Vision-Language Models

Peng Xia, Siwei Han, Shi Qiu, Yiyang Zhou, Zhaoyang Wang, Wenhao Zheng, Zhaorun Chen, Chenhang Cui, Mingyu Ding, Linjie Li, Lijuan Wang, Huaxiu Yao

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments ICLR 2025 Oral

Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models

Jiaqi Li, Qianshan Wei, Chuanyi Zhang, Guilin Qi, Miaozeng Du, Yongrui Chen, Sheng Bi, Fan Liu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Oasis: One Image is All You Need for Multimodal Instruction Data Synthesis

Letian Zhang, Quan Cui, Bingchen Zhao, Cheng Yang

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimization

Zefeng Zhang, Hengzhu Tang, Jiawei Sheng, Zhenyu Zhang, Yiming Ren, Zhenyang Li, Dawei Yin, Duohe Ma, Tingwen Liu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments CVPR 2025

SeqAfford: Sequential 3D Affordance Reasoning via Multimodal Large Language Model

Chunlin Yu, Hanqing Wang, Ye Shi, Haoyang Luo, Sibei Yang, Jingyi Yu, Jingya Wang

专题命中 多模态评测 ：multimodal(title)；multi-modal(abstract)；分类 cs.CV、cs.AI

ViBe: A Text-to-Video Benchmark for Evaluating Hallucination in Large Multimodal Models

Vipula Rawte, Sarthak Jain, Aarush Sinha, Garv Kaushik, Aman Bansal, Prathiksha Rumale Vishwanath, Samyak Rajesh Jain, Aishwarya Naresh Reganti, Vinija Jain, Aman Chadha, Amit P. Sheth, Amitava Das

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Real-world validation of a multimodal LLM-powered pipeline for High-Accuracy Clinical Trial Patient Matching leveraging EHR data

Anatole Callies, Quentin Bodinier, Philippe Ravaud, Kourosh Davarpanah

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

Leveraging Large Language Model-based Room-Object Relationships Knowledge for Enhancing Multimodal-Input Object Goal Navigation

Leyuan Sun, Asako Kanezaki, Guillaume Caron, Yusuke Yoshiyasu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Comments will soon submit to the Elsevier journal, Advanced Engineering Informatics

Journal ref Advanced Engineering Informatics 65 (2025)

MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process Errors Identification

Zhaopan Xu, Pengfei Zhou, Jiaxin Ai, Wangbo Zhao, Kai Wang, Xiaojiang Peng, Wenqi Shao, Hongxun Yao, Kaipeng Zhang

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.AI

Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models

Jiacong Xu, Shao-Yuan Lo, Bardia Safaei, Vishal M. Patel, Isht Dwivedi

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments 19 pages, 10 figures, accepted by CVPR 2025

Cascaded Self-Evaluation Augmented Training for Lightweight Multimodal LLMs

Zheqi Lv, Wenkai Wang, Jiawei Wang, Shengyu Zhang, Fei Wu

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CL、cs.AI

SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models

Haotian Xia, Zhengbang Yang, Junbo Zou, Rhys Tracy, Yuqing Wang, Chi Lu, Christopher Lai, Yanjun He, Xun Shao, Zhuoqing Xie, Yuan-fang Wang, Weining Shen, Hanjie Chen

专题命中 多模态评测 ：multimodal(title,abstract)；分类 cs.CV、cs.CL

Comments ICLR 2025 Poster