Dimba：Transformer-Mamba扩散模型
Dimba: Transformer-Mamba Diffusion Models
AI总结：
本文提出Dimba，一种结合Transformer和Mamba层的混合架构文本到图像扩散模型，通过交替堆叠和交叉注意力整合条件信息，在图像质量、艺术渲染和语义控制上与基准相当，同时显著提升吞吐量并降低内存占用。
摘要：
本文推出了Dimba，一种新型文本到图像扩散模型，它采用结合Transformer和Mamba元素的独特混合架构。具体而言，Dimba的顺序堆叠块在Transformer和Mamba层之间交替，并通过交叉注意力层整合条件信息，从而利用两种架构范式的优势。我们研究了几种优化策略，包括质量调优、分辨率适配，并确定了大规模图像生成所需的关键配置。该模型的灵活设计支持满足特定资源约束和目标的场景。在适当扩展时，与传统的纯基于Transformer的基准相比，Dimba提供了显著的吞吐量并减少了内存占用。大量实验表明，Dimba在图像质量、艺术渲染和语义控制方面实现了与基准相当的性能。我们还报告了评估期间发现的架构的几个有趣性质，并在实验中发布了检查点。我们的发现强调了大规模混合Transformer-Mamba架构在扩散模型基础阶段的潜力，预示着文本到图像生成的光明未来。
英文摘要：
This paper unveils Dimba, a new text-to-image diffusion model that employs a distinctive hybrid architecture combining Transformer and Mamba elements. Specifically, Dimba sequentially stacked blocks alternate between Transformer and Mamba layers, and integrate conditional information through the cross-attention layer, thus capitalizing on the advantages of both architectural paradigms. We investigate several optimization strategies, including quality tuning, resolution adaption, and identify critical configurations necessary for large-scale image generation. The model's flexible design supports scenarios that cater to specific resource constraints and objectives. When scaled appropriately, Dimba offers substantial throughput and a reduced memory footprint relative to conventional pure Transformers-based benchmarks. Extensive experiments indicate that Dimba achieves comparable performance compared with benchmarks in terms of image quality, artistic rendering, and semantic control. We also report several intriguing properties of architecture discovered during evaluation and release checkpoints in experiments. Our findings emphasize the promise of large-scale hybrid Transformer-Mamba architectures in the foundational stage of diffusion models, suggesting a bright future for text-to-image generation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
