Though Multi-modal Large Language Models (MLLMs) have recently achieved significant progress, they often struggle to understand diverse and complicated inter-object relations. Specifically, the lack of large-scale and high-quality relation data has greatly hindered the progress of MLLMs in various vision-language perception tasks. We attempt to address this challenge by contributing the Multi-Modal Relation Understanding benchmark (MMRel), which features large-scale, high-quality, and diverse data on inter-object relations. MMRel has three distinctive attributes: (i) it contains 22,500 question-answer pairs spanning three distinct domains and around 400 relations, ensuring both scale and diversity; (ii) it provides manually verified, high-quality labels to ensure exceptional annotation accuracy; and (iii) it includes adversarial cases with highly unusual relations, offering a challenging setting for evaluating relation hallucination. These features make MMRel ideal for evaluating MLLMs on relation understanding, as well as for fine-tuning MLLMs to enhance relation comprehension capability. Extensive experiments on 28 MLLMs demonstrate the effectiveness of MMRel in both evaluating and enhancing MLLMs' relation understanding, and the accompanying analyses provide insights for future research. The benchmark has been made publicly available at: https://niejiahao1998.github.io/MMRel
MUSE: Benchmarking Large Vision-Language Models on Multi-Modal Understanding in Situated Education
AI Singapore, National University of Singapore（新加坡国立大学新加坡人工智能）；School of Computing, National University of Singapore（新加坡国立大学计算学院）；Institute of Advanced Intelligence and Computing, A*STAR（新加坡科技研究局先进智能与计算研究所）
Omni-Persona: Systematic Benchmarking and Improving Omnimodal Personalization
Department of Electrical and Computer Engineering, Seoul National University（首尔国立大学电气与计算机工程系）；Interdisciplinary Program in Artificial Intelligence, Seoul National University（首尔国立大学人工智能跨学科项目）；Department of Artificial Intelligence, University of Seoul（首尔大学人工智能系）
SnapBench: Benchmarking Snap-and-Ask Multimodal Retrieval for Mobile Interactions
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；Tencent Yuanbao（腾讯元宝）；Tsinghua University（清华大学）；ARC Lab, Tencent（腾讯ARC实验室）；The University of Hong Kong（香港大学）
ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation
University of Maryland, College Park（马里兰大学学院公园分校）；University of Pennsylvania（宾夕法尼亚大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；University of Michigan（密歇根大学）；University of Southern California（南加州大学）
Q-Doc: Benchmarking Document Image Quality Assessment Capabilities in Multi-modal Large Language Models
South China Normal University（华南师范大学）；Nanyang Technological University（南洋理工大学）；City University of Hong Kong（香港城市大学）；Shenzhen Academy of Inspection and Quarantine（深圳检验检疫局）；Shenzhen University（深圳大学）
Cross-Modal Benchmarking for Robotic Perception in Natural Environments
CSIRO Robotics, CSIRO, Australia（CSIRO机器人研究所，CSIRO，澳大利亚）；University of Sydney (USyd), Australia（悉尼大学（USyd），澳大利亚）；Queensland University of Technology (QUT), Australia（昆士兰理工大学（QUT），澳大利亚）
PhytoSynth: Leveraging Multi-modal Generative Models for Crop Disease Data Generation with Novel Benchmarking and Prompt Engineering Approach
Gulf Coast Research and Education Center (GCREC), University of Florida（佛罗里达大学 Gulf Coast Research and Education Center）；Citrus Research and Education Center (CREC), University of Florida（佛罗里达大学 Citrus Research and Education Center）
SciMMIR: Benchmarking Scientific Multi-modal Information Retrieval
Multimodal Art Projection Research Community（多模态艺术投影研究共同体）；University of Manchester（曼彻斯特大学）；University of Waterloo（滑铁卢大学）；Hong Kong University of Science and Technology（香港科技大学）；Durham University（杜伦大学）
MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents
INSAIT；Lanzhou University（兰州大学）；King Abdullah University of Science and Technology（国王阿卜杜勒-阿齐兹科学与技术大学）；Heriot-Watt University（赫瑞-沃德大学）；Monash University（莫纳什大学）；University College Dublin（都柏林大学学院）
VideoMathQA: Benchmarking Mathematical Reasoning via Multimodal Understanding in Videos
MBZUAI（穆桑大学人工智能研究所）；University of California Merced（加州大学默塞德分校）；Google Research（谷歌研究）；Australian National University（澳大利亚国立大学）；Linköping University（林肯大学）
UniDiff: Parameter-Efficient Adaptation of Diffusion Models for Land Cover Classification with Multi-Modal Remotely Sensed Imagery and Sparse Annotations
WEAVE: Unleashing and Benchmarking the In-context Interleaved Comprehension and Generation
National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）；University of Maryland, College Park（马里兰大学学院公园分校）；Zhejiang University（浙江大学）
TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos
University of Electronic Science and Technology of China（电子科学与技术大学）；Peking University（北京大学）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Zhongguancun Academy（中关村学院）
The S3LI Vulcano Dataset: A Dataset for Multi-Modal SLAM in Unstructured Planetary Environments
German Aerospace Center (DLR) Institute of Robotics and Mechatronics（德国航空航天中心（DLR）机器人与机电研究所）；KTH Royal Institute of Technology（皇家理工学院）；Deptartment of Intelligent Systems（智能系统部门）
Multi-Modal Explainable Medical AI Assistant for Trustworthy Human-AI Collaboration
Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology（电子与计算机工程系，香港科学与技术大学）；Department of Computer Science and Engineering, The Hong Kong University of Science and Technology（计算机科学与工程系，香港科学与技术大学）
