EVLM:面向跨维度视觉编辑的自反思多模态推理
EVLM: Self-Reflective Multimodal Reasoning for Cross-Dimensional Visual Editing
AI总结:
EVLM 提出结合参考视觉与自反思推理框架,并通过 RKTO 对齐将模糊用户意图转化为精确编辑提示,在图像、视频、3D 和 4D 编辑任务中显著提升与人类意图的一致性。
摘要:
从模糊或部分指定的指令中编辑复杂视觉内容仍然是视觉-语言建模中的核心挑战。现有模型能够对内容进行语境化处理,但往往无法推断参考图像或场景中的潜在意图,从而导致编辑结果不一致或错位。我们提出了编辑视觉-语言模型(EVLM),该系统结合参考视觉来解释模糊指令,以生成精确且具备上下文感知能力的编辑提示。EVLM 的关键创新在于一个反思性推理框架,该框架通过反思感知 KL 散度目标优化(RKTO)与人工评分的理由对齐,将主观用户意图转化为结构化、可执行的输出。通过将思维链(CoT)推理与 RKTO 对齐相结合,EVLM 在不依赖二元监督的情况下捕捉细粒度的编辑偏好。EVLM 在一个包含 30,000 个带有人工标注理由质量的 CoT 示例的数据集上进行训练,在与人类意图的对齐方面取得了显著提升。跨图像、视频、3D 和 4D 编辑任务的实验表明,EVLM 能够生成连贯且高质量的指令,为多模态编辑和推理提供了可扩展的基础。
英文摘要:
Editing complex visual content from ambiguous or partially specified instructions remains a core challenge in vision-language modeling. Existing models can contextualize content but often fail to infer the underlying intent within a reference image or scene, leading to inconsistent or misaligned edits. We introduce the Editing Vision-Language Model (EVLM), a system that interprets ambiguous instructions in conjunction with reference visuals to produce precise, context-aware editing prompts. EVLM's key innovation is a reflective reasoning framework that translates subjective user intent into structured, actionable outputs by aligning with human-rated rationales through Reflection-Aware KL-Divergence Target Optimization (RKTO). By combining Chain-of-Thought (CoT) reasoning with RKTO alignment, EVLM captures fine-grained editing preferences without relying on binary supervision. Trained on a dataset of 30,000 CoT examples with human-annotated rationale quality, EVLM achieves substantial gains in alignment with human intent. Experiments across image, video, 3D, and 4D editing tasks show that EVLM generates coherent and high-quality instructions, providing a scalable foundation for multimodal editing and reasoning.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。