MME-VideoOCR：评估多模态大语言模型在视频场景中基于OCR的能力
MME-VideoOCR: Evaluating OCR-Based Capabilities of Multimodal LLMs in Video Scenarios
AI总结：
本文推出MME-VideoOCR视频OCR基准，评估18个最先进MLLMs的视频OCR能力，发现现有模型在需视频整体理解的任务中表现有限，最优模型准确率仅73.7%，明确了高分辨率输入等对视频OCR的重要性。
摘要：
多模态大语言模型（MLLMs）在静态图像光学字符识别（OCR）方面已达到相当高的准确率，但受视频内容固有的运动模糊、时间变化、视觉效果等因素影响，其在视频OCR任务中的效能显著下降。为给实用型MLLMs的训练提供更清晰指导，本文推出MME-VideoOCR基准，涵盖全面的视频OCR应用场景。该基准包含10个任务类别、25个独立任务，覆盖44种不同场景，任务范围超出文本识别，还涉及视频中文本内容的深度理解与推理；由1464个分辨率、宽高比、时长各异的视频，以及2000条精心筛选、人工标注的问答对组成。研究在MME-VideoOCR上评估了18个当前最先进的MLLMs，结果显示即使是性能最优的Gemini-2.5 Pro，准确率也仅为73.7%。细粒度分析表明，现有MLLMs在相关文本包含于单帧或少数帧的任务中表现较强，但在需要对视频进行整体理解的任务中能力有限，在涉及时空推理、跨帧信息整合或需抵御语言先验偏差的场景中，这些局限尤为明显。研究还强调，高分辨率视觉输入与充足时间覆盖对动态视频场景中可靠OCR的重要性。
英文摘要：
Multimodal Large Language Models (MLLMs) have achieved considerable accuracy in Optical Character Recognition (OCR) from static images. However, their efficacy in video OCR is significantly diminished due to factors such as motion blur, temporal variations, and visual effects inherent in video content. To provide clearer guidance for training practical MLLMs, we introduce the MME-VideoOCR benchmark, which encompasses a comprehensive range of video OCR application scenarios. MME-VideoOCR features 10 task categories comprising 25 individual tasks and spans 44 diverse scenarios. These tasks extend beyond text recognition to incorporate deeper comprehension and reasoning of textual content within videos. The benchmark consists of 1,464 videos with varying resolutions, aspect ratios, and durations, along with 2,000 meticulously curated, manually annotated question-answer pairs. We evaluate 18 state-of-the-art MLLMs on MME-VideoOCR, revealing that even the best-performing model (Gemini-2.5 Pro) achieves an accuracy of only 73.7%. Fine-grained analysis indicates that while existing MLLMs demonstrate strong performance on tasks where relevant texts are contained within a single or few frames, they exhibit limited capability in effectively handling tasks that demand holistic video comprehension. These limitations are especially evident in scenarios that require spatio-temporal reasoning, cross-frame information integration, or resistance to language prior bias. Our findings also highlight the importance of high-resolution visual input and sufficient temporal coverage for reliable OCR in dynamic video scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
