UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Models
University of California San Diego（加利福尼亚大学圣迭戈分校）；University of Southern California（南加利福尼亚大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Carnegie Mellon University（卡内基梅隆大学）
Cheng Yang, Chufan Shi, Bo Shui, Yaokang Wu, Muzi Tao, Huijuan Wang, Ivan Yee Lee, Yong Liu, Xuezhe Ma, Taylor Berg-Kirkpatrick
Unified multimodal models (UMMs) aim to integrate multimodal understanding and generation within a unified architecture, yet it remains unclear to what extent textual and visual modalities are aligned. To investigate this question, we use reasoning-guided image generation as a diagnostic task, where models produce textual reasoning first and then generate images. We introduce UReason, a benchmark for evaluating reasoning-to-generation alignment in this paradigm, consisting of 2,000 human-curated and human-verified instances spanning five reasoning-intensive tasks: Code, Arithmetic, Spatial, Attribute, and Text. To enable controlled analysis, we develop an evaluation framework that compares direct generation, reasoning-guided generation, and decontextualized generation, which conditions only on the refined prompt extracted from reasoning. Across eight widely used open-source UMMs, while we find that reasoning-guided generation yields improvements over direct generation, somewhat surprisingly, decontextualized generation consistently outperforms reasoning-guided generation by a large margin. Our further analyses suggest that the intended visual semantics in textual reasoning are not reliably reflected in the generated images, despite their unified design and training. Overall, UReason serves as a practical litmus test for reasoning-to-generation alignment and provides a challenging benchmark for developing next-generation, more tightly aligned UMMs.
UniIR: Training and Benchmarking Universal Multimodal Information Retrievers
University of Waterloo（滑铁卢大学）；Georgia Institute of Technology（佐治亚理工学院）；Hong Kong University of Science and Technology（香港科技大学）；Google DeepMind（谷歌DeepMind）
MulTaBench: Benchmarking Multimodal Tabular Learning with Text and Image
Technion – Israel Institute of Technology（技术ion – 以色列理工学院）；Prior Labs（Prior实验室）；NVIDIA；SODA Team, INRIA Saclay, Palaiseau（SODA团队，INRIA萨克莱，帕莱索）；University of Freiburg（弗赖堡大学）；Probabl；ELLIS Institute Tübingen（图宾根ELLIS研究所）
MMSciBench: Benchmarking Language Models on Chinese Multimodal Scientific Problems
Department of Computer Science, Yale University（耶鲁大学计算机科学系）；Department of Computer Science, Brown University（布朗大学计算机科学系）；School of Data Science, Fudan University（复旦大学数据科学学院）；Shanghai Key Laboratory of Data Science（上海数据科学关键实验室）；Datawiz LLC
MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University（多媒体可信感知与高效计算重点实验室，中国教育部，厦门大学）；Zhongguancun Academy, Beijing, China（中关村学院，北京，中国）
Emotion and Intent Joint Understanding in Multimodal Conversation: A Benchmarking Dataset
Inner Mongolia University（内蒙古大学）；Chinese Academy of Sciences（中国科学院）；South China University of Technology（华南理工大学）；Technical University of Munich（慕尼黑工业大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））
School of Computer Science and Engineering, NTU（南洋理工大学计算机科学与工程学院）；School of Natural Sciences, University of Stirling（斯特灵大学自然科学学院）；Temasek Laboratories, NTU（南洋理工大学淡马锡实验室）；National Institute of Technology, Warangal（瓦朗加尔国立理工学院）
M$^3$Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions
The Hong Kong University of Science and Technology（香港科学与技术大学）；Beijing University of Chemical Technology（北京化工大学）；The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州））；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））；Beijing Institute of Technology (Zhuhai)（北京理工大学（珠海））；Tencent Hy（腾讯（深圳））；Peng Cheng Laboratory（鹏城实验室）
UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents
School of Computer Science and Engineering, Northeastern University, Shenyang, China（东北大学计算机科学与工程学院）；Department of Computer Science and Technology, Tsinghua University, Beijing, China（清华大学计算机科学与技术系）；Alibaba Group, Hangzhou, China（阿里巴巴集团）
MathScape: Benchmarking Multimodal Large Language Models in Real-World Mathematical Contexts
Peking University（北京大学）；University of Chinese Academy of Sciences（中国科学院大学）；Nankai University（南开大学）；Beijing Institute of Technology（北京理工大学）；Baichuan Inc.（百度文心）
CODIS: Benchmarking Context-Dependent Visual Comprehension for Multimodal Large Language Models
Tsinghua University（清华大学）；Jilin University（吉林大学）；Northwest Minzu University（西北民族大学）；Institute for AI Industry Research (AIR), Tsinghua University（清华大学人工智能产业研究院）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）
Mem-Gallery: Benchmarking Multimodal Long-Term Conversational Memory for MLLM Agents
University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；MIT-IBM Watson AI Lab, IBM Research（MIT-IBM沃森人工智能实验室，IBM研究）；Stony Brook University（石溪大学）；Brookhaven National Laboratory（布鲁赫斯国家实验室）
I Came, I Saw, I Explained: Benchmarking Multimodal LLMs on Figurative Meaning in Memes
MaiNLP, Center for Information and Language Processing, LMU Munich（MaiNLP，信息与语言处理中心，慕尼黑大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；National Research Council Canada（加拿大国家研究委员会）
MMCode: Benchmarking Multimodal Large Language Models for Code Generation with Visually Rich Programming Problems
National University of Singapore（新加坡国立大学）；The University of Hong Kong（香港大学）；Nanyang Technological University（南洋理工大学）；Hong Kong Baptist University（香港浸会大学）
Can't See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs
Renmin University of China（中国人民大学）；Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Chinese University of Hong Kong（香港中文大学）；Johns Hopkins University（约翰霍普金斯大学）；Hong Kong University of Science and Technology（香港科技大学）；Tencent（腾讯）
