画布到图像：利用多模态控制的组合图像生成
Canvas-to-Image: Compositional Image Generation with Multimodal Controls
AI总结：
Canvas-to-Image通过统一框架整合多模态控制，提升图像生成的组合性和控制精度。
摘要：
尽管现代扩散模型在生成高质量和多样化的图像方面表现出色，但它们仍然在高保真的组合和多模态控制方面存在困难，尤其是在用户同时指定文本提示、主体参考、空间布局、姿态约束和布局注释时。我们引入了Canvas-to-Image，一个统一的框架，将这些异质控制整合到一个单一的画布界面中，使用户能够生成忠实反映其意图的图像。我们的核心思想是将多样化的控制信号编码到一个单一的复合画布图像中，使模型能够直接解读以实现整合的视觉-空间推理。我们进一步整理了一套多任务数据集，并提出了一种多任务画布训练策略，优化扩散模型以在统一的学习范式下联合理解和整合异质控制到文本到图像生成中。这种联合训练使Canvas-to-Image能够在多种控制模态之间推理，而不是依赖于任务特定的启发式方法，并且在推理过程中能够很好地推广到多控制场景中。广泛的实验表明，Canvas-to-Image在挑战性基准测试中，如多人物组合、姿态控制组合、布局受限生成和多控制生成中，在身份保持和控制遵循方面显著优于最先进的方法。
英文摘要：
While modern diffusion models excel at generating high-quality and diverse images, they still struggle with high-fidelity compositional and multimodal control, particularly when users simultaneously specify text prompts, subject references, spatial arrangements, pose constraints, and layout annotations. We introduce Canvas-to-Image, a unified framework that consolidates these heterogeneous controls into a single canvas interface, enabling users to generate images that faithfully reflect their intent. Our key idea is to encode diverse control signals into a single composite canvas image that the model can directly interpret for integrated visual-spatial reasoning. We further curate a suite of multi-task datasets and propose a Multi-Task Canvas Training strategy that optimizes the diffusion model to jointly understand and integrate heterogeneous controls into text-to-image generation within a unified learning paradigm. This joint training enables Canvas-to-Image to reason across multiple control modalities rather than relying on task-specific heuristics, and it generalizes well to multi-control scenarios during inference. Extensive experiments show that Canvas-to-Image significantly outperforms state-of-the-art methods in identity preservation and control adherence across challenging benchmarks, including multi-person composition, pose-controlled composition, layout-constrained generation, and multi-control generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
