重新思考富上下文布局到图像生成的训练与评估
Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation
AI总结：
针对布局到图像生成中复杂文本描述表征不足及训练评估设置不匹配的问题，提出新型区域交叉注意力模块和两个开放词汇评估指标，并通过用户研究验证指标与人类偏好的一致性。
摘要：
生成模型的最新进展显著提升了其图像生成能力，催生了图像编辑、补全和视频编辑等广泛应用。布局到图像（L2I）生成是生成建模中的一个专门领域，其中预定义的对象布局引导生成过程。本研究提出了一种专为丰富布局到图像生成设计的新型区域交叉注意力模块。该模块显著改善了布局区域的表征，尤其是在现有方法难以应对高度复杂且详细的文本描述的场景中。此外，当前的开放词汇L2I方法虽在开放集设置下训练，但其评估往往在闭集环境中进行。为弥补这一差距，我们提出了两个指标来评估开放词汇场景下的L2I性能。另外，我们开展了全面的用户研究，以验证这些指标与人类偏好的一致性。
英文摘要：
Recent advancements in generative models have significantly enhanced their capacity for image generation, enabling a wide range of applications such as image editing, completion and video editing. A specialized area within generative modeling is layout-to-image (L2I) generation, where predefined layouts of objects guide the generative process. In this study, we introduce a novel regional cross-attention module tailored to enrich layout-to-image generation. This module notably improves the representation of layout regions, particularly in scenarios where existing methods struggle with highly complex and detailed textual descriptions. Moreover, while current open-vocabulary L2I methods are trained in an open-set setting, their evaluations often occur in closed-set environments. To bridge this gap, we propose two metrics to assess L2I performance in open-vocabulary scenarios. Additionally, we conduct a comprehensive user study to validate the consistency of these metrics with human preferences.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
