Omni IIE Bench：图像编辑模型实际能力的基准测试
Omni IIE Bench: Benchmarking the Practical Capabilities of Image Editing Models
AI总结：
本文提出Omni IIE Bench，通过双轨诊断设计评估图像编辑模型在不同语义尺度任务中的一致性，揭示模型在低到高语义任务间性能显著下降的问题。
摘要：
尽管基于指令的图像编辑（IIE）已取得显著进展，现有基准通过混合评估追求任务广度，这种范式掩盖了专业应用中关键的失败模式：模型在不同语义尺度的任务间表现不一致。为解决这一差距，我们引入Omni IIE Bench，一个高质量、人工标注的基准，专门用于诊断IIE模型在实际应用中的编辑一致性。Omni IIE Bench包含创新的双轨诊断设计：（1）单轮一致性，包含属性修改和实体替换的共享上下文任务对；（2）多轮协调，涉及连续对话任务，跨越语义尺度。该基准通过极其严格的多阶段人工过滤过程构建，包含由计算机视觉研究生执行的质量标准和由专业设计师进行的行业相关性审查。我们使用Omni IIE Bench对8种主流IIE模型进行了全面评估。我们的分析首次量化了一个普遍存在的性能差距：几乎所有模型在从低语义尺度任务过渡到高语义尺度任务时都表现出显著的性能下降。Omni IIE Bench为下一代更可靠、更稳定的IIE模型开发提供了关键的诊断工具和见解。
英文摘要：
While Instruction-based Image Editing (IIE) has achieved significant progress, existing benchmarks pursue task breadth via mixed evaluations. This paradigm obscures a critical failure mode crucial in professional applications: the inconsistent performance of models across tasks of varying semantic scales. To address this gap, we introduce Omni IIE Bench, a high-quality, human-annotated benchmark specifically designed to diagnose the editing consistency of IIE models in practical application scenarios. Omni IIE Bench features an innovative dual-track diagnostic design: (1) Single-turn Consistency, comprising shared-context task pairs of attribute modification and entity replacement; and (2) Multi-turn Coordination, involving continuous dialogue tasks that traverse semantic scales. The benchmark is constructed via an exceptionally rigorous multi-stage human filtering process, incorporating a quality standard enforced by computer vision graduate students and an industry relevance review conducted by professional designers. We perform a comprehensive evaluation of 8 mainstream IIE models using Omni IIE Bench. Our analysis quantifies, for the first time, a prevalent performance gap: nearly all models exhibit a significant performance degradation when transitioning from low-semantic-scale to high-semantic-scale tasks. Omni IIE Bench provides critical diagnostic tools and insights for the development of next-generation, more reliable, and stable IIE models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
