OCRGenBench: 一个全面的评估OCR生成能力的基准
OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilities
AI总结：
本文提出OCRGenBench，通过整合文本中心的T2I生成、文本编辑和OCR相关的图像到图像翻译任务，全面评估模型的视觉文本综合能力，揭示现有模型在文本定位、内容修改和密集文本处理上的不足。
摘要：
改进视觉文本合成长期以来一直是图像生成模型面临的挑战。尽管最近最先进的模型在文本生成能力上取得了显著进展，但现有基准由于范围狭窄（仅限场景文本和海报）、孤立评估（T2I生成或编辑分开）和难度不足（缺乏挑战性场景）而无法准确评估其真实性能。为弥合这一差距，我们首次将以文本为中心的T2I生成、文本编辑和OCR相关的图像到图像翻译任务统一起来，以评估模型的综合视觉文本合成能力，即OCR生成能力。因此，我们提出了OCRGenBench，目前最全面的评估这些能力的基准。OCRGenBench涵盖五种常见文本类别和33种OCR生成任务，包括T2I生成、文本编辑和其他图像到图像OCR任务（如文档去斜和手写去除）。该基准包含1060个由人类标注的样本，包含指令-图像-GT三元组，故意具有高文本密度、多样化的生成规模、多样的长宽比和双语内容，以捕捉现实世界的复杂性。此外，我们引入了OCRGenScore，一个整合文本准确度、审美质量和遵循指示的统一度量标准。对19种最先进的生成模型进行的广泛实验表明，大多数得分低于60/100。我们的分析揭示了关键但此前被忽视的限制，包括文本定位差、意外内容修改和对密集或小规模文本的失败。我们希望OCRGenBench建立一个稳健的标准来评估OCR生成能力，推动可靠视觉文本合成的发展。该基准和评估代码可在https://github.com/NiceRingNode/Awesome-Generative-Models-for-OCR上获得。
英文摘要：
Improving visual text synthesis has long been a challenging and evolving frontier for image generation models. While recent state-of-the-art (SOTA) models have made remarkable strides in text generation capabilities, existing benchmarks inadequately assess their true performance due to narrow scope (scene text and posters only), isolated evaluation (T2I generation or editing separately), and insufficient difficulty (lacking challenging scenarios). To bridge this gap, we pioneer the unification of text-centric T2I generation, text editing, and OCR-related image-to-image translation to evaluate a model's holistic visual text synthesis abilities, i.e., OCR generative capabilities. Accordingly, we propose OCRGenBench, the most comprehensive benchmark to date for evaluating these abilities. OCRGenBench covers five common text categories and 33 OCR generative tasks, encompassing T2I generation, text editing, and other image-to-image OCR tasks (e.g., document dewarping and handwriting removal). The benchmark includes 1,060 human-annotated samples consisting of instruction-image-GT triplets, deliberately featuring high text density, diverse generation scales, varied aspect ratios, and bilingual content to capture real-world complexity. Furthermore, we introduce OCRGenScore, a unified metric integrating text accuracy, aesthetic quality, and instruction following. Extensive experiments on 19 cutting-edge generative models reveal that most score below 60/100. Our analysis exposes critical, previously overlooked limitations, including poor text localization, unintended content modifications, and failures with dense or small-scale text. We hope OCRGenBench establishes a robust standard to evaluate OCR generative capabilities, driving the evolution of reliable visual text synthesis. The benchmark and evaluation code are available at https://github.com/NiceRingNode/Awesome-Generative-Models-for-OCR.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
