MSRAMIE：多模态结构推理代理用于多指令图像编辑
MSRAMIE: Multimodal Structured Reasoning Agent for Multi-instruction Image Editing
AI总结：
MSRAMIE基于多模态大语言模型构建无需训练的代理框架，通过结构化多模态推理处理多指令图像编辑任务，提升复杂指令遵循度和完成概率，同时保持图像质量和一致性。
摘要：
现有基于指令的图像编辑模型在简单单步指令下表现良好，但在涉及多个、长且相互依赖的指令的现实场景中性能下降。主要原因是缺乏具有复杂多指令注释的训练数据。然而，收集此类数据并重新训练这些模型成本高昂。为此，我们提出了MSRAMIE，一个基于多模态大语言模型（MLLM）的免训练代理框架。MSRAMIE将现有编辑模型作为插件组件，通过结构化多模态推理处理多指令任务。它通过MLLM-based Instructor和图像编辑Actor之间的迭代交互进行协调，引入了提出的状态树和参考图拓扑结构。在推理过程中，复杂指令被分解为多个编辑步骤，使状态转换、跨步骤信息聚合和原始输入回忆成为可能，从而系统地探索图像编辑空间并实现灵活的逐步输出细化。可可视化推理拓扑进一步提供了可解释和可控的决策路径。实验表明，随着指令复杂性增加，MSRAMIE在指令遵循方面提高了超过15%，并在单次运行中完成所有修改的概率增加了超过100%，同时保持感知质量和视觉一致性。
英文摘要：
Existing instruction-based image editing models perform well with simple, single-step instructions but degrade in realistic scenarios that involve multiple, lengthy, and interdependent directives. A main cause is the scarcity of training data with complex multi-instruction annotations. However, it is costly to collect such data and retrain these models. To address this challenge, we propose MSRAMIE, a training-free agent framework built on Multimodal Large Language Model (MLLM). MSRAMIE takes existing editing models as plug-in components and handle multi-instruction tasks via structured multimodal reasoning. It orchestrates iterative interactions between an MLLM-based Instructor and an image editing Actor, introducing a novel reasoning topology that comprises the proposed Tree-of-States and Graph-of-References. During inference, complex instructions are decomposed into multiple editing steps which enable state transitions, cross-step information aggregation, and original input recall, which enables systematic exploration of the image editing space and flexible progressive output refinement. The visualizable inference topology further provides interpretable and controllable decision pathways. Experiments show that as the instruction complexity increases, MSRAMIE can improve instruction following over 15% and increases the probability of finishing all modifications in a single run over 100%, while preserving perceptual quality and maintaining visual consistency.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
