EditCaption: 用于图像编辑指令合成的人工精炼SFT与HAE-DPO
EditCaption: Human-Refined SFT and HAE-DPO for Image Editing Instruction Synthesis
AI总结：
提出EditCaption两阶段后训练流程，通过人工精炼SFT和基于难度自适应错误感知DPO（HAE-DPO）提升图像编辑指令合成质量，显著降低关键错误率并超越现有模型。
摘要：
高质量的源-目标图像对及精确的编辑指令对于指令引导的图像编辑至关重要，但大规模构建此类训练三元组成本高昂。最近的流程通常依赖视觉语言模型自动合成编辑指令，但我们发现强大的VLM仍难以描述图像对之间的视觉变换。具体而言，它们表现出三种反复出现的失败模式：方向不一致、视角模糊和缺少细粒度属性。在400个图像对的人工评估中，多个开源VLM基线产生超过47%的关键错误率，使得许多合成指令不适合下游训练。为解决此问题，我们提出EditCaption，一种用于图像编辑指令合成的两阶段后训练流程。首先，通过基于GLM的自动字幕生成、EditScore过滤和人工精炼构建100K监督微调数据集。其次，收集10K人工标注的偏好对，其中每个被拒绝的指令都标注了其主要错误类型和严重程度。基于此数据集，我们提出难度自适应错误感知DPO（HAE-DPO），一种任务适配的DPO目标，它引入了基于人工标注的严重程度、失败模式类型和参考模型难度的自适应边界。在三个基准上的实验表明，我们的235B模型经过SFT+HAE-DPO后在开源和闭源模型中达到最先进性能，在Eval-400、HQ-Edit和ByteMorph-Bench上分别获得4.720、4.672和4.651分——在所有三个基准上均超越Gemini-3-Pro。人工评估证实关键错误率从47.75%降至17.50%，正确率从41.75%提升至70.25%，超越Gemini-3-Pro（66.00%）。
英文摘要：
High-quality source-target image pairs with precise editing instructions are essential for instruction-guided image editing, yet constructing such training triplets at scale remains costly. Recent pipelines often rely on vision-language models to synthesize editing instructions automatically, but we find that strong VLMs still struggle to describe visual transformations between image pairs. In particular, they exhibit three recurring failure modes: orientation inconsistency, viewpoint ambiguity, and missing fine-grained attributes. In a human evaluation on 400 image pairs, several open-source VLM baselines produce critical-error rates above 47\%, making many synthesized instructions unsuitable for downstream training. To address this, we propose EditCaption, a two-stage post-training pipeline for image editing instruction synthesis. First, we construct a 100K supervised fine-tuning dataset through GLM-based auto-captioning, EditScore filtering, and human refinement. Second, we collect 10K human-annotated preference pairs, where each rejected instruction is labeled with its primary error type and severity. Based on this dataset, we propose Hardness-Adaptive Error-Aware DPO (HAE-DPO), a task-adapted DPO objective that introduces an adaptive margin based on human-labeled severity, failure-mode type, and reference-model hardness. Experiments across three benchmarks demonstrate that our 235B model with SFT+HAE-DPO achieves state-of-the-art performance among open-source and closed models, scoring 4.720 on Eval-400, 4.672 on HQ-Edit, and 4.651 on ByteMorph-Bench -- surpassing Gemini-3-Pro on all three. Human evaluation confirms critical error rates drop from 47.75\% to 17.50\%, with correct rates improving from 41.75\% to 70.25\%, surpassing Gemini-3-Pro (66.00\%).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
