SIGMA: 带多属性标记的选择性交错生成
SIGMA: Selective-Interleaved Generation with Multi-Attribute Tokens
AI总结:
SIGMA通过引入多属性标记实现多条件生成,提升可控性、一致性和视觉质量,优于Bagel在组合任务中的表现。
摘要:
最近的统一模型,如Bagel,表明配对的图像编辑数据可以有效地在单个扩散变换器中对齐多个视觉任务。然而,这些模型仍然局限于单条件输入,并缺乏合成来自多个异质来源结果的灵活性。我们提出了SIGMA(带多属性标记的选择性交错生成),一种统一的后训练框架,使扩散变换器能够实现交错的多条件生成。SIGMA引入了选择性的多属性标记,包括风格、内容、主体和身份标记,这些标记使模型能够在交错的文本-图像序列中解释和组合多个视觉条件。通过在Bagel统一主干上进行后训练,使用700,000个交错示例,SIGMA支持组合编辑、选择性属性转移和细粒度多模态对齐。广泛的实验表明,SIGMA在多样化的编辑和生成任务中提高了可控性、跨条件一致性和视觉质量,与Bagel相比在组合任务上取得了显著优势。
英文摘要:
Recent unified models such as Bagel demonstrate that paired image-edit data can effectively align multiple visual tasks within a single diffusion transformer. However, these models remain limited to single-condition inputs and lack the flexibility needed to synthesize results from multiple heterogeneous sources. We present SIGMA (Selective-Interleaved Generation with Multi-Attribute Tokens), a unified post-training framework that enables interleaved multi-condition generation within diffusion transformers. SIGMA introduces selective multi-attribute tokens, including style, content, subject, and identity tokens, which allow the model to interpret and compose multiple visual conditions in an interleaved text-image sequence. Through post-training on the Bagel unified backbone with 700K interleaved examples, SIGMA supports compositional editing, selective attribute transfer, and fine-grained multimodal alignment. Extensive experiments show that SIGMA improves controllability, cross-condition consistency, and visual quality across diverse editing and generation tasks, with substantial gains over Bagel on compositional tasks.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。