ZoomLDM：用于多尺度图像生成的潜在扩散模型
ZoomLDM: Latent Diffusion Model for multi-scale image generation
AI总结：
针对大型图像生成中现有补丁模型无法捕捉全局结构的问题，提出多尺度潜在扩散模型ZoomLDM，利用自监督嵌入的放大感知条件机制，实现跨尺度高质量生成，支持4096×4096像素图像和4倍超分辨率。
摘要：
扩散模型已经彻底改变了图像生成领域，然而若干挑战限制了其在大型图像领域（如数字病理学和卫星图像）中的应用。鉴于在这些可能达到千兆像素大小的领域中，直接在“整幅”图像上训练模型是不可行的，基于扩散的生成方法一直专注于合成从这些图像中提取的小型、固定大小的补丁。然而，生成小补丁的适用性有限，因为基于补丁的模型无法捕捉大型图像的全局结构和更广泛的上下文，而这对于合成（语义上）准确的样本可能至关重要。为了克服这一限制，我们提出了ZoomLDM，一种专门用于跨多个尺度生成图像的扩散模型。我们方法的核心是一种新颖的放大感知条件机制，该机制利用自监督学习（SSL）嵌入，并允许扩散模型在不同“缩放”级别（即从大型图像中在不同尺度提取的固定大小补丁）合成图像。ZoomLDM能够合成连贯的组织病理学图像，这些图像在不同缩放级别下保持上下文准确和细节丰富，在所有尺度上实现了最先进的图像生成质量，并在生成整个大型图像缩略图的数据稀缺场景中表现出色。ZoomLDM的多尺度特性解锁了大型图像生成的额外能力，实现了计算上可行且全局连贯的图像合成，最高可达$4096 \times 4096$像素以及$4\times$超分辨率。此外，从ZoomLDM提取的多尺度特征在多实例学习实验中非常有效。
英文摘要：
Diffusion models have revolutionized image generation, yet several challenges restrict their application to large-image domains, such as digital pathology and satellite imagery. Given that it is infeasible to directly train a model on 'whole' images from domains with potential gigapixel sizes, diffusion-based generative methods have focused on synthesizing small, fixed-size patches extracted from these images. However, generating small patches has limited applicability since patch-based models fail to capture the global structures and wider context of large images, which can be crucial for synthesizing (semantically) accurate samples. To overcome this limitation, we present ZoomLDM, a diffusion model tailored for generating images across multiple scales. Central to our approach is a novel magnification-aware conditioning mechanism that utilizes self-supervised learning (SSL) embeddings and allows the diffusion model to synthesize images at different 'zoom' levels, i.e., fixed-size patches extracted from large images at varying scales. ZoomLDM synthesizes coherent histopathology images that remain contextually accurate and detailed at different zoom levels, achieving state-of-the-art image generation quality across all scales and excelling in the data-scarce setting of generating thumbnails of entire large images. The multi-scale nature of ZoomLDM unlocks additional capabilities in large image generation, enabling computationally tractable and globally coherent image synthesis up to $4096 \times 4096$ pixels and $4\times$ super-resolution. Additionally, multi-scale features extracted from ZoomLDM are highly effective in multiple instance learning experiments.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
