arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2607.00920cs.CV

GMO-E$^2$DIT:面向电商图像的接地多操作编辑

GMO-E$^2$DIT: Grounded Multi-Operation Editing for E-Commerce Images

Wuhan University(武汉大学);Xi’an Jiaotong University(西安交通大学);Sun Yat-sen University(中山大学)

Zipeng Guo, Xiaoan Liu, Lichen Ma, Cheng Wang, Yu He, Xiaolong Fu, Jingling Fu, Xinyuan Shan, Shaojie Guo, Luohang Liu, Junshi Huang, Yan Li

AI总结:

提出GMO-E$^2$DIT框架,通过VLM代理构建区域接地编辑议程,结合掩码条件图像编辑器和反思循环,实现多操作、可审计的电商图像编辑,在指令准确性和编辑保真度上超越现有基线。

摘要:

现实世界的电商图像编辑通常需要多个、局部化且可审计的操作,而非全局风格化。这种组合性质带来了双重挑战:模型必须精确地将所有请求的编辑应用到正确的区域,同时保留未修改的内容,即使在模糊指令下也是如此。现有的一次性编辑器将意图解析、空间接地和合成合并为单一步骤,经常导致部分执行失败,这在商业场景中是不可接受的。为了解决这个问题,我们引入了GMO-E$^2$DIT,一个代理编辑框架,它将视觉语言模型(VLM)与掩码条件图像编辑器相结合,以处理结构化的多轮任务完成。给定一个不明确的指令,VLM代理构建一个区域接地的编辑议程,有效地将认知推理与生成渲染解耦。然后,该框架通过操作感知掩码和参考执行子程序,利用反思驱动循环检查中间结果并确定后续状态。这种迭代机制可靠地保留安全的局部进展,重试未完成的操作,并从错误中恢复。此外,我们开发了一个统一的数据管道,为规划、执行和反思提供对齐的监督,以及EComEditBench,一个用于指令驱动评估的综合基准。大量实验表明,GMO-E$^2$DIT与强大的闭源模型相比取得了竞争性性能,在指令准确性和编辑保真度上优于现有基线。

英文摘要:

Real-world e-commerce image editing often requires multiple, localized, and auditable operations rather than global restyling. This compositional nature poses a dual challenge: models must precisely apply all requested edits to the correct regions while preserving unmodified content, even under ambiguous instructions. Existing one-shot editors conflate intent resolution, spatial grounding, and synthesis into a single step, frequently resulting in partial execution failures, which is unacceptable for commercial scenarios. To address this, we introduce GMO-E$^2$DIT, an agentic editing framework that couples a Vision-Language Model (VLM) with a mask-conditioned image editor to tackle structured multi-turn task completion. Given an underspecified instruction, the VLM agent constructs a region-grounded edit agenda, effectively decoupling cognitive reasoning from generative rendering. The framework then executes sub-programs via operation-aware masks and references, utilizing a reflection-driven loop to inspect intermediate results and determine the subsequent state. This iterative mechanism reliably preserves safe partial progress, retries unfinished operations, and recovers from errors. Furthermore, we develop a unified data pipeline providing aligned supervision for planning, execution, and reflection, alongside EComEditBench, a comprehensive benchmark for instruction-driven evaluation. Extensive experiments demonstrate that GMO-E$^2$DIT achieves competitive performance compared to strong closed-source models, yielding superior instruction accuracy and edit fidelity over existing baselines.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑