强化学习在自回归图像编辑中的应用前景
The Promise of RL for Autoregressive Image Editing
AI总结：
针对文本引导图像编辑任务的执行难题，本文对比三种性能提升策略，提出结合RL与多模态LLM验证器的图像编辑模型EARL，用更少训练数据取得有竞争力的性能并开源相关资源。
摘要：
尽管当前图像生成技术已能够生成符合多句子提示词的高质量图像，但文本引导的图像编辑任务仍是一项挑战，即便仅包含几个词的编辑请求也常常无法被正确执行。本文探索了三种提升各类图像编辑任务性能的策略：监督微调（SFT）、强化学习（RL）以及思维链（CoT）推理。为在统一框架下研究上述所有组件，本文采用了以统一方式处理文本和视觉token的自回归多模态模型。研究发现，强化学习结合大型多模态LLM验证器是上述策略中效果最优的。基于此，本文发布了EARL（基于自回归与强化学习的编辑模型），这是一个性能强劲的基于强化学习的图像编辑模型，尽管其使用的训练数据远少于强基线模型，但在各类编辑任务上的表现具备竞争力，推动了自回归多模态模型在图像编辑领域的前沿发展。本文已在https://github.com/mair-lab/EARL公开代码、训练数据及训练好的模型。
英文摘要：
While image generation techniques are now capable of producing high-quality images that respect prompts which span multiple sentences, the task of text-guided image editing remains a challenge. Even edit requests that consist of only a few words often fail to be executed correctly. We explore three strategies to enhance performance on a wide range of image editing tasks: supervised fine-tuning (SFT), reinforcement learning (RL), and Chain-of-Thought (CoT) reasoning. In order to study all these components in one consistent framework, we adopt an autoregressive multimodal model that processes textual and visual tokens in a unified manner. We find RL combined with a large multi-modal LLM verifier to be the most effective of these strategies. As a result, we release EARL: Editing with Autoregression and RL, a strong RL-based image editing model that performs competitively on a diverse range of edits compared to strong baselines, despite using much less training data. Thus, EARL pushes the frontier of autoregressive multimodal models on image editing. We release our code, training data, and trained models at https://github.com/mair-lab/EARL.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
