赋能视觉创造力：面向图像编辑推荐的视觉-语言助手
Empowering Visual Creativity: A Vision-Language Assistant to Image Editing Recommendations
AI总结：
本文提出图像编辑推荐（IER）任务，并构建Creativity-VLA多模态框架，通过专用数据集训练及'token-for-localization'机制支持全局与局部编辑，自动生成与输入图像和用户提示高度相关的创意编辑指令。
摘要：
基于文本的图像生成与编辑的进步彻底改变了内容创作，使用户能够从富有想象力的文本提示中创建令人印象深刻的内容。然而，现有方法在处理典型场景中常见的过于简化的提示时效果不佳，这些场景通常发生在用户开始编辑时只有模糊或抽象的目的。这些场景需要用户付出精心的构思努力，以弥合这种模糊起点与描绘所需结果所需的详细创意想法之间的差距。在本文中，我们引入了图像编辑推荐（IER）任务。该任务旨在从输入图像和代表用户未明确指定编辑目的的简单提示中，自动生成多样化的创意编辑指令。为此，我们引入了Creativity-Vision Language Assistant (Creativity-VLA)，这是一个专为编辑指令生成设计的多模态框架。我们在专门为IER策划的编辑指令数据集上训练Creativity-VLA。我们进一步通过一种新颖的'token-for-localization'机制增强了我们的模型，使其能够支持全局和局部编辑操作。我们的实验结果证明了\ours{}在建议指令方面的有效性，这些建议指令不仅包含引人入胜的创意元素，而且与输入图像和用户的初始提示保持高度相关。
英文摘要：
Advances in text-based image generation and editing have revolutionized content creation, enabling users to create impressive content from imaginative text prompts. However, existing methods are not designed to work well with the oversimplified prompts that are often encountered in typical scenarios when users start their editing with only vague or abstract purposes in mind. Those scenarios demand elaborate ideation efforts from the users to bridge the gap between such vague starting points and the detailed creative ideas needed to depict the desired results. In this paper, we introduce the task of Image Editing Recommendation (IER). This task aims to automatically generate diverse creative editing instructions from an input image and a simple prompt representing the users' under-specified editing purpose. To this end, we introduce Creativity-Vision Language Assistant~(Creativity-VLA), a multimodal framework designed specifically for edit-instruction generation. We train Creativity-VLA on our edit-instruction dataset specifically curated for IER. We further enhance our model with a novel 'token-for-localization' mechanism, enabling it to support both global and local editing operations. Our experimental results demonstrate the effectiveness of \ours{} in suggesting instructions that not only contain engaging creative elements but also maintain high relevance to both the input image and the user's initial hint.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
