arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.05518cs.HCcs.CV

CoEditor++:基于指令的视觉编辑 via 认知推理

CoEditor++: Instruction-based Visual Editing via Cognitive Reasoning

Hong Kong Polytechnic University(香港理工大学);Harbin Institute of Technology(哈尔滨工业大学);Microsoft(微软公司)

Minheng Ni, Yutao Fan, Zhengyuan Yang, Yeli Shen, Yuxiang Wei, Yaowen Zhang, Lijuan Wang, Lei Zhang, Wangmeng Zuo

AI总结:

CoEditor++通过认知推理实现基于指令的视觉编辑,无需训练即可在通用和负责任的编辑任务中取得最佳性能。

摘要:

最近大型多模态模型(LMMs)的进步使基于指令的图像编辑成为可能,允许用户通过自然语言描述修改视觉内容。然而,现有方法在高层语义推理和视觉一致性方面往往存在问题,尤其是在模糊或复杂的指令下。为了解决这些挑战,我们提出了CoEditor++,一个认知结构化、无需训练的框架,通过两个认知阶段和反思性自我选择机制将编辑分解为

英文摘要:

Recent advances in large multimodal models (LMMs) have enabled instruction-based image editing, allowing users to modify visual content via natural language descriptions. However, existing approaches often struggle with high-level semantic reasoning and visual consistency, particularly under ambiguous or complex instructions. To address these challenges, we propose CoEditor++, a cognitively structured, training-free framework that decomposes editing into "what to edit" and "how to edit" through two cognitive stages with a reflective self-selection mechanism, enabling robust, fine-grained, and interpretable editing. Built entirely from open-sourced components, CoEditor++ requires no additional training or fine-tuning, ensuring transparency and cross-domain applicability. We evaluate CoEditor++ on SmartEdit, a widely used benchmark for general editing, and AltBear, a privacy and compliance-oriented benchmark. Experimental results show that CoEditor++ achieves state-of-the-art performance in both general editing and responsible editing tasks compared with open-sourced models that require training on specialized editing datasets maintaining significantly higher visual consistency. When compared with closed-source models such as Nano Banana Pro or GPT-4o, CoEditor++ preserves comparable instruction following while still substantially outperforming them in visual consistency. Extensive ablation studies confirm that the effectiveness of CoEditor++ benefits from its structured cognitive design rather than any specific model component. Our findings suggest the potential toward cognitive-centric instruction-based image editing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑