EventBench: 向事件驱动的大语言模型全面评估迈进
EventBench: Towards Comprehensive Benchmarking of Event-based MLLMs
AI总结：
EventBench通过八个任务指标和大规模数据集全面评估事件驱动MLLMs的能力，揭示其在细粒度识别和空间推理方面的不足。
摘要：
多模态大语言模型（MLLMs）在事件驱动视觉领域取得了显著进展，但对其能力进行全面评估的统一基准仍鲜有探索。本文引入EventBench，该基准提供八个多样化的任务指标以及大规模事件流数据集。EventBench与现有事件驱动基准相比有四个关键区别：（1）在可访问性上开放，释放所有原始事件流和八个评估指标的任务指令；（2）任务覆盖多样性，涵盖理解、识别和空间推理任务，以全面评估能力；（3）在空间维度上整合，开创了为事件驱动MLLMs设计的3D空间推理任务；（4）数据量规模，配套超过一百万对事件-文本数据集，支持大规模训练和评估。使用EventBench，我们评估了最先进的闭源模型，如GPT-5和Gemini-2.5 Pro，以及开源模型，如Qwen2.5-VL和InternVL3，以及直接处理原始事件流的事件驱动MLLMs，如EventGPT。广泛评估表明，尽管当前事件驱动MLLMs在事件流理解方面表现强劲，但在细粒度识别和空间推理方面仍存在困难。
英文摘要：
Multimodal large language models (MLLMs) have made significant advancements in event-based vision, yet the comprehensive evaluation of their capabilities within a unified benchmark remains largely unexplored. In this work, we introduce EventBench, a benchmark that offers eight diverse task metrics together with a large-scale event stream dataset. EventBench differs from existing event-based benchmarks in four key aspects: (1) openness in accessibility, releasing all raw event streams and task instructions across eight evaluation metrics; (2) diversity in task coverage, spanning understanding, recognition, and spatial reasoning tasks for comprehensive capability assessment; (3) integration in spatial dimensions, pioneering the design of 3D spatial reasoning tasks for event-based MLLMs; and (4) scale in data volume, with an accompanying training set of over one million event-text pairs supporting large-scale training and evaluation. Using EventBench, we evaluate state-of-the-art closed-source models such as GPT-5 and Gemini-2.5 Pro, leading open-source models including Qwen2.5-VL and InternVL3, and event-based MLLMs such as EventGPT that directly process raw event streams. Extensive evaluation reveals that while current event-based MLLMs demonstrate strong performance in event stream understanding, they continue to struggle with fine-grained recognition and spatial reasoning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
