模型能多好地遵循视觉指令？VIBE：一个系统性的视觉指令驱动图像编辑基准
How Well Do Models Follow Visual Instructions? VIBE: A Systematic Benchmark for Visual Instruction-Driven Image Editing
AI总结：
本文提出VIBE基准，用于评估视觉指令驱动的图像编辑模型，通过三级交互层次评估指涉 grounding、形态操作和因果推理，并发现专有模型在早期阶段表现优异但随着任务难度增加性能下降。
摘要：
最近的生成模型在图像编辑方面取得了显著进展。然而，现有系统和基准仍然主要是文本引导的。相比之下，人类交流本质上是多模态的，视觉指令如草图能高效传达空间和结构意图。为填补这一差距，我们引入VIBE，即图像编辑的视觉指令基准，其三级交互层次捕捉了指涉 grounding、形态操作和因果推理。在这些层次中，我们精心挑选了高质量且多样的测试用例，反映了视觉指令遵循的逐步增加的复杂性。我们进一步提出一个稳健的LMM-as-a-judge评估框架，配有任务特定的指标，以实现可扩展且细致的评估。通过全面评估17个代表性的开源和专有图像编辑模型，我们发现专有模型在早期阶段展现出视觉指令遵循能力，并且一贯优于开源模型。然而，随着任务难度的增加，即使是最强的系统性能也会显著下降，这揭示了未来研究的有希望方向。
英文摘要：
Recent generative models have achieved remarkable progress in image editing. However, existing systems and benchmarks remain largely text-guided. In contrast, human communication is inherently multimodal, where visual instructions such as sketches efficiently convey spatial and structural intent. To address this gap, we introduce VIBE, the Visual Instruction Benchmark for Image Editing with a three-level interaction hierarchy that captures deictic grounding, morphological manipulation, and causal reasoning. Across these levels, we curate high-quality and diverse test cases that reflect progressively increasing complexity in visual instruction following. We further propose a robust LMM-as-a-judge evaluation framework with task-specific metrics to enable scalable and fine-grained assessment. Through a comprehensive evaluation of 17 representative open-source and proprietary image editing models, we find that proprietary models exhibit early-stage visual instruction-following capabilities and consistently outperform open-source models. However, performance degrades markedly with increasing task difficulty even for the strongest systems, highlighting promising directions for future research.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
