基于推理的编辑：基于假设指令的图像编辑与视觉推理
Reasoning to Edit: Hypothetical Instruction-Based Image Editing with Visual Reasoning
AI总结：
本文提出Reason50K数据集和ReasonBrain框架，通过多模态大语言模型和扩散模型实现复杂隐含指令的图像编辑，提升视觉推理能力。
摘要：
基于推理的图像编辑（IIE）随着扩散模型的成功而迅速发展。然而，现有研究主要集中在简单且明确的指令上，以执行添加、删除、移动或交换对象等编辑操作。它们难以处理更复杂的隐含假设性指令，这些指令需要更深入的推理来推断合理的视觉变化和用户意图。此外，当前数据集对训练和评估具有推理意识的编辑能力提供有限的支持。在架构上，这些方法也缺乏支持此类推理的细粒度细节提取机制。为了解决这些限制，我们提出了Reason50K，一个专门用于训练和评估假设性指令推理图像编辑的大型数据集，以及ReasonBrain，一个全新的框架，旨在在多种场景中推理并执行隐含假设性指令。Reason50K包含超过50,000个样本，涵盖四个关键推理场景：物理、时间、因果和故事推理。ReasonBrain利用多模态大语言模型（MLLMs）生成编辑指导，并利用扩散模型进行图像合成，结合细粒度推理提示提取（FRCE）模块，以捕捉支持指令推理的详细视觉和文本语义。为了减少语义损失，我们进一步引入了跨模态增强器（CME），以促进细粒度提示与MLLM衍生特征之间的丰富交互。广泛的实验表明，ReasonBrain在推理场景中一致优于最先进的基线，在零样本泛化到传统IIE任务时也表现出色。我们的数据集和代码将公开发布。
英文摘要：
Instruction-based image editing (IIE) has advanced rapidly with the success of diffusion models. However, existing efforts primarily focus on simple and explicit instructions to execute editing operations such as adding, deleting, moving, or swapping objects. They struggle to handle more complex implicit hypothetical instructions that require deeper reasoning to infer plausible visual changes and user intent. Additionally, current datasets provide limited support for training and evaluating reasoning-aware editing capabilities. Architecturally, these methods also lack mechanisms for fine-grained detail extraction that support such reasoning. To address these limitations, we propose Reason50K, a large-scale dataset specifically curated for training and evaluating hypothetical instruction reasoning image editing, along with ReasonBrain, a novel framework designed to reason over and execute implicit hypothetical instructions across diverse scenarios. Reason50K includes over 50K samples spanning four key reasoning scenarios: Physical, Temporal, Causal, and Story reasoning. ReasonBrain leverages Multimodal Large Language Models (MLLMs) for editing guidance generation and a diffusion model for image synthesis, incorporating a Fine-grained Reasoning Cue Extraction (FRCE) module to capture detailed visual and textual semantics essential for supporting instruction reasoning. To mitigate the semantic loss, we further introduce a Cross-Modal Enhancer (CME) that enables rich interactions between the fine-grained cues and MLLM-derived features. Extensive experiments demonstrate that ReasonBrain consistently outperforms state-of-the-art baselines on reasoning scenarios while exhibiting strong zero-shot generalization to conventional IIE tasks. Our dataset and code will be released publicly.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
