从静态到动态:基于物理的图像编辑与潜在过渡先验
From Statics to Dynamics: Physics-Aware Image Editing with Latent Transition Priors
AI总结:
本文提出PhysicEdit,通过引入物理状态过渡和大规模数据集,提升图像编辑的物理真实性和知识引导能力,达到开源方法的新水平。
摘要:
基于指令的图像编辑在语义对齐方面取得了显著成功,但最先进的模型在涉及复杂因果动态(如折射或材料变形)的编辑任务中,往往无法生成物理上合理的结果。我们归因于主流范式将编辑视为图像对之间的离散映射,仅提供边界条件,而未明确指定过渡动态。为此,我们重新表述物理感知的编辑为预测物理状态过渡,并引入PhysicTran38K,一个包含38K条过渡轨迹的大型视频数据集,涵盖五个物理领域,通过两阶段过滤和约束感知标注流程构建。基于此监督,我们提出PhysicEdit,一个端到端框架,配备文本-视觉双思机制。它结合冻结的Qwen2.5-VL用于物理基础推理,与可学习的过渡查询相结合,为扩散主干提供时间步适应的视觉引导。实验表明,PhysicEdit在物理真实性和知识引导编辑方面比Qwen-Image-Edit分别提升5.9%和10.1%,为开源方法设定了新的状态-of-the-art,同时在领先专有模型中保持竞争力。
英文摘要:
Instruction-based image editing has achieved remarkable success in semantic alignment, yet state-of-the-art models frequently fail to render physically plausible results when editing involves complex causal dynamics, such as refraction or material deformation. We attribute this limitation to the dominant paradigm that treats editing as a discrete mapping between image pairs, which provides only boundary conditions and leaves transition dynamics underspecified. To address this, we reformulate physics-aware editing as predictive physical state transitions and introduce PhysicTran38K, a large-scale video-based dataset comprising 38K transition trajectories across five physical domains, constructed via a two-stage filtering and constraint-aware annotation pipeline. Building on this supervision, we propose PhysicEdit, an end-to-end framework equipped with a textual-visual dual-thinking mechanism. It combines a frozen Qwen2.5-VL for physically grounded reasoning with learnable transition queries that provide timestep-adaptive visual guidance to a diffusion backbone. Experiments show that PhysicEdit improves over Qwen-Image-Edit by 5.9% in physical realism and 10.1% in knowledge-grounded editing, setting a new state-of-the-art for open-source methods, while remaining competitive with leading proprietary models.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。