VDE Bench: 评估图像编辑模型对视觉文档进行修改的能力
VDE Bench: Evaluating The Capability of Image Editing Models to Modify Visual Documents
AI总结：
本文提出VDE Bench，一个专门评估图像编辑模型在双语中文-英文和复杂视觉文档编辑任务性能的基准，通过高质量数据集和新的评估框架，系统量化了文本修改的准确性。
摘要：
近年来，图像编辑模型取得了显著进展，使用户能够通过自然语言指令灵活地交互式地操作视觉内容。然而，一个重要的但尚未充分研究的研究方向是密集的视觉文档图像编辑，这涉及在图像中修改文本内容，同时忠实保留原始文本风格和背景上下文。现有方法主要集中在英语场景和文本相对稀疏的图像上，因此无法充分解决密集、结构复杂的文档或非拉丁文字如中文。为弥合这一差距，我们提出了VDE Bench（视觉文档编辑基准），这是一个严格人工标注和评估的基准，专门设计用于评估图像编辑模型在双语中文-英文和复杂视觉文档编辑任务上的性能。该基准包含942个基于指令的图像编辑样本数据集，其种子图像涵盖密集的中文和英文文本文档，包括学术论文、海报、演示文稿、考试材料和报纸。此外，我们引入了一个新的评估框架，系统地量化了在OCR解析层面的编辑性能，从而实现了对文本修改准确性的细粒度评估。基于此基准，我们对代表性图像编辑模型进行了全面评估。人类验证显示，人类判断与自动化评估指标之间有一致性。VDE Bench构成了评估图像编辑模型在双语密集文本视觉文档性能的首个系统性基准。
英文摘要：
In recent years, image editing models have made significant progress, enabling users to manipulate visual content in a flexible and interactive manner through natural language instructions. However, an important yet underexplored research direction remains dense visual document image editing, which involves modifying textual content within images while faithfully preserving the original text style and background context. Existing methods primarily focus on English scenarios and images with relatively sparse text, and thus cannot adequately address dense, structurally complex documents or non-Latin scripts such as Chinese. To bridge this gap, we propose VDE Bench (Visual Doc Edit Bench), a rigorously human annotated and evaluated benchmark specifically designed to assess the performance of image editing models on bilingual Chinese-English and complex visual document editing tasks. The benchmark comprises a high quality dataset of 942 instruction based image editing samples, whose seed images encompass dense Chinese and English text documents including academic papers, posters, presentation slides, examination materials, and newspapers. Furthermore, we introduce a novel evaluation framework that systematically quantifies editing performance at the OCR parsing level, thereby enabling fine grained assessment of text modification accuracy. Based on this benchmark, we conduct a comprehensive evaluation of representative image editing models. Human verification demonstrates a high degree of consistency between human judgments and automated evaluation metrics. VDE Bench constitutes the first systematic benchmark for evaluating the performance of image editing models on bilingual dense text visual documents.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
