事件定制图像生成
Event-Customized Image Generation
AI总结：
本文提出FreeEvent方法，通过引入实体切换和事件转移路径，实现事件定制化图像生成，提升复杂场景下的定制化能力。
摘要：
定制化图像生成，即根据用户指定的概念生成定制化图像，因其创造性和新颖性而受到广泛关注。尽管在主体定制方面取得了显著进展，一些开创性工作进一步探索了动作和交互的定制化，超越了实体（即人类、动物和物体）的外观。然而，这些方法仅关注两个实体之间的基本动作和交互，其效果受到缺乏“完全相同”的参考图像的限制。为了将定制化图像生成扩展到更复杂的场景以满足一般现实应用的需求，我们提出了一项新的任务：事件定制化图像生成。给定一张单张参考图像，我们将“事件”定义为场景中不同实体之间所有特定动作、姿势、关系或交互。该任务旨在准确捕捉复杂的事件并生成具有各种目标实体的定制化图像。为了解决这一任务，我们提出了一种新的无训练事件定制化方法：FreeEvent。具体而言，FreeEvent在通用扩散去噪过程中引入了两条额外路径：1）实体切换路径：它对目标实体生成应用交叉注意力引导和调节。2）事件转移路径：它将参考图像的空间特征和自注意力图注入到目标图像中以生成事件。为进一步促进这一新任务，我们收集了两个评估基准：SWiG-Event和Real-Event。广泛的实验和消融实验已证明了FreeEvent的有效性。
英文摘要：
Customized Image Generation, generating customized images with user-specified concepts, has raised significant attention due to its creativity and novelty. With impressive progress achieved in subject customization, some pioneer works further explored the customization of action and interaction beyond entity (i.e., human, animal, and object) appearance. However, these approaches only focus on basic actions and interactions between two entities, and their effects are limited by insufficient ''exactly same'' reference images. To extend customized image generation to more complex scenes for general real-world applications, we propose a new task: event-customized image generation. Given a single reference image, we define the ''event'' as all specific actions, poses, relations, or interactions between different entities in the scene. This task aims at accurately capturing the complex event and generating customized images with various target entities. To solve this task, we proposed a novel training-free event customization method: FreeEvent. Specifically, FreeEvent introduces two extra paths alongside the general diffusion denoising process: 1) Entity switching path: it applies cross-attention guidance and regulation for target entity generation. 2) Event transferring path: it injects the spatial feature and self-attention maps from the reference image to the target image for event generation. To further facilitate this new task, we collected two evaluation benchmarks: SWiG-Event and Real-Event. Extensive experiments and ablations have demonstrated the effectiveness of FreeEvent.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
