arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.14186cs.CV
收藏

UniRef-Image-Edit: 向可扩展和一致的多参考图像编辑迈进

UniRef-Image-Edit: Towards Scalable and Consistent Multi-Reference Image Editing

Tsinghua University(清华大学);Kuaishou Technology(快手科技);Hong Kong Polytechnic University(香港理工大学);Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳));CUHK MMLab(香港中文大学MMLab)
Hongyang Wei, Bin Wen, Yancheng Long, Yankai Yang, Yuhang Hu, Tianke Zhang, Wei Chen, Haonan Fan, Kaiyu Jiang, Jiankang Chen, Changyi Liu, Kaiyu Tang, Haojie Di… 展开作者

Hongyang Wei, Bin Wen, Yancheng Long, Yankai Yang, Yuhang Hu, Tianke Zhang, Wei Chen, Haonan Fan, Kaiyu Jiang, Jiankang Chen, Changyi Liu, Kaiyu Tang, Haojie Ding, Xiao Yang, Jia Sun, Huaiqing Wang, Zhenyu Yang, Xinyu Wei, Xianglong He, Yangguang Li, Fan Yang, Tingting Gao, Lei Zhang, Guorui Zhou, Han Li

AI总结:

UniRef-Image-Edit通过统一的输入表示和两阶段训练框架,提升多参考图像编辑的一致性和保真度。

摘要:

我们提出了UniRef-Image-Edit,一种高性能的多模态生成系统,它在一个框架内统一了单图像编辑和多图像合成。现有的扩散基编辑方法往往难以在多个条件下保持一致性,因为参考输入之间的交互有限。为了解决这个问题,我们引入了序列扩展潜在融合(SELF),一种统一的输入表示,能够动态地将多个参考图像序列化为一个连贯的潜在序列。在专门的训练阶段,所有参考图像都被联合约束以适应固定长度的序列,同时满足全局像素预算约束。基于SELF,我们提出了一种两阶段训练框架,包括监督微调(SFT)和强化学习(RL)。在SFT阶段,我们联合训练单图像编辑和多图像合成任务,以建立稳健的生成先验。我们采用逐步序列长度训练策略,其中所有输入图像最初被调整为总像素预算为1024²,然后逐步增加到1536²和2048²,以提高视觉保真度和跨参考一致性。这种逐步放松压缩使模型能够逐步捕捉更细致的视觉细节,同时保持参考之间的稳定对齐。对于RL阶段,我们引入了多源GRPO(MSGRPO),据我们所知,这是首个为多参考图像生成量身定制的强化学习框架。MSGRPO优化模型以协调冲突的视觉约束,显著提高了合成的一致性。我们将会开源代码、模型、训练数据和奖励数据,供社区研究使用。

英文摘要:

We present UniRef-Image-Edit, a high-performance multi-modal generation system that unifies single-image editing and multi-image composition within a single framework. Existing diffusion-based editing methods often struggle to maintain consistency across multiple conditions due to limited interaction between reference inputs. To address this, we introduce Sequence-Extended Latent Fusion (SELF), a unified input representation that dynamically serializes multiple reference images into a coherent latent sequence. During a dedicated training stage, all reference images are jointly constrained to fit within a fixed-length sequence under a global pixel-budget constraint. Building upon SELF, we propose a two-stage training framework comprising supervised fine-tuning (SFT) and reinforcement learning (RL). In the SFT stage, we jointly train on single-image editing and multi-image composition tasks to establish a robust generative prior. We adopt a progressive sequence length training strategy, in which all input images are initially resized to a total pixel budget of $1024^2$, and are then gradually increased to $1536^2$ and $2048^2$ to improve visual fidelity and cross-reference consistency. This gradual relaxation of compression enables the model to incrementally capture finer visual details while maintaining stable alignment across references. For the RL stage, we introduce Multi-Source GRPO (MSGRPO), to our knowledge the first reinforcement learning framework tailored for multi-reference image generation. MSGRPO optimizes the model to reconcile conflicting visual constraints, significantly enhancing compositional consistency. We will open-source the code, models, training data, and reward data for community research purposes.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑