基于布局的可控病理图像生成与上下文扩散变换器
Layout-Guided Controllable Pathology Image Generation with In-Context Diffusion Transformers
AI总结：
本文提出基于布局的可控病理图像生成方法，通过多代理LVLM注释框架构建精细临床对齐的监督数据，提出IC-DiT模型整合空间布局、文本描述和视觉嵌入，实现高保真生成与强空间可控性。
摘要：
可控病理图像合成需要可靠调节空间布局、组织形态和语义细节。然而，现有文本引导的扩散模型仅提供粗略全局控制，缺乏强制细粒度结构约束的能力。进展受限于缺乏将patch级空间布局与详细诊断描述配对的大规模数据集，因为为gigapixel全片图像生成此类注释对人类专家来说过于耗时。为克服这些挑战，我们首先开发了一个可扩展的多代理LVLM注释框架，将图像描述、诊断步骤提取和自动质量判断整合到协调的流程中，并通过人类验证过程评估系统的可靠性。该框架能够高效构建大规模的细粒度和临床对齐的监督数据。基于精心整理的数据，我们提出In-Context Diffusion Transformer (IC-DiT)，一种具有布局意识的生成模型，将空间布局、文本描述和视觉嵌入整合到统一的扩散变换器中。通过层次化多模态注意力，IC-DiT在保持全局语义一致性的同时，准确保留结构和形态细节。在五个组织病理学数据集上的广泛实验表明，IC-DiT在保真度、空间可控性和诊断一致性方面均优于现有方法。此外，生成的图像还为下游任务如癌症分类和生存分析提供了有效的数据增强资源。
英文摘要：
Controllable pathology image synthesis requires reliable regulation of spatial layout, tissue morphology, and semantic detail. However, existing text-guided diffusion models offer only coarse global control and lack the ability to enforce fine-grained structural constraints. Progress is further limited by the absence of large datasets that pair patch-level spatial layouts with detailed diagnostic descriptions, since generating such annotations for gigapixel whole-slide images is prohibitively time-consuming for human experts. To overcome these challenges, we first develop a scalable multi-agent LVLM annotation framework that integrates image description, diagnostic step extraction, and automatic quality judgment into a coordinated pipeline, and we evaluate the reliability of the system through a human verification process. This framework enables efficient construction of fine-grained and clinically aligned supervision at scale. Building on the curated data, we propose In-Context Diffusion Transformer (IC-DiT), a layout-aware generative model that incorporates spatial layouts, textual descriptions, and visual embeddings into a unified diffusion transformer. Through hierarchical multimodal attention, IC-DiT maintains global semantic coherence while accurately preserving structural and morphological details. Extensive experiments on five histopathology datasets show that IC-DiT achieves higher fidelity, stronger spatial controllability, and better diagnostic consistency than existing methods. In addition, the generated images serve as effective data augmentation resources for downstream tasks such as cancer classification and survival analysis.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
