Scarcity of annotated data, particularly for rare or atypical morphologies, present significant challenges for cell and nuclei segmentation in computational pathology. While manual annotation is labor-intensive and costly, synthetic data offers a cost-effective alternative. We introduce a Multimodal Semantic Diffusion Model (MSDM) for generating realistic pixel-precise image-mask pairs for cell and nuclei segmentation. By conditioning the generative process with cellular/nuclear morphologies (using horizontal and vertical maps), RGB color characteristics, and BERT-encoded assay/indication metadata, MSDM generates datasests with desired morphological properties. These heterogeneous modalities are integrated via multi-head cross-attention, enabling fine-grained control over the generated images. Quantitative analysis demonstrates that synthetic images closely match real data, with low Wasserstein distances between embeddings of generated and real images under matching biological conditions. The incorporation of these synthetic samples, exemplified by columnar cells, significantly improves segmentation model accuracy on columnar cells. This strategy systematically enriches data sets, directly targeting model deficiencies. We highlight the effectiveness of multimodal diffusion-based augmentation for advancing the robustness and generalizability of cell and nuclei segmentation models. Thereby, we pave the way for broader application of generative models in computational pathology.
Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models
AIRI;HSE University(俄罗斯高等经济大学);Simula Metropolitan Center for Digital Engineering(Simula 数字工程研究中心);Oslo Metropolitan University(奥斯陆 Metropolitan 大学);Kazan Federal University(卡扎ン联邦大学)
Bi-parametric prostate MR image synthesis using pathology and sequence-conditioned stable diffusion
University College London(伦敦大学学院);City University of Hong Kong(香港城市大学);University College Hospital NHS foundation Trust(大学学院医院国民保健服务基金会信托)
我们提出了一种图像合成机制,用于以文本为条件的多序列前列腺MR图像,以控制病变存在和序列,以及生成以图像为条件的配对双参数图像,例如用于从配对数据的T2加权MR生成扩散加权MR,这是病理图像合成中的两个挑战性任务。我们提出的机制利用并建立在最近的稳定扩散模型,提出基于图像的条件配对数据生成。我们验证我们的方法使用2D图像切片从真正的疑似前列腺癌患者。合成图像的真实性通过盲态专家评估进行验证,以识别真实图像与假图像,其中具有4年泌尿科MR阅读经验的放射科医生在所有测试序列中仅达到59.4%的准确度(概率为50%)。对于第一次,我们通过盲专家识别疑似病变的存在来评估所生成的病理的真实性,我们发现临床医生对真实图像和合成图像的表现相似,真实图像和合成图像之间的病变识别准确性差异为2.9个百分点,证明了放射学培训目的的潜力。此外,我们还表明,经过病变识别训练的机器学习模型在使用合成数据增强的真实数据进行训练时,表现出更好的性能(76.2% vs 70.4%,统计学显著改善),而不是仅使用真实图像进行训练,这证明了模型训练的有用性。
PathoSyn: Imaging-Pathology MRI Synthesis via Disentangled Deviation Diffusion
Department of Radiology, Boston Children’s Hospital, Harvard Medical School(放射科,波士顿儿童医院,哈佛医学院);College of Science, Northeastern University(科学学院,东北大学);School of Medicine, Yale University(医学院,耶鲁大学);Department of Cardiac Surgery, The Second Affiliated Hospital of Jiangxi Medical College, Nanchang University(心脏外科科,江西医学院第二附属医院,南昌大学);College of Computing and Data Science, Nanyang Technological University(计算与数据科学学院,南洋理工大学)
Patch-to-Global: Random Patch Diffusion for Globally Consistent Megapixel Artifact Inpainting in Whole Slide Images
Korea University College of Medicine(高丽大学医学院);Korea University Anam Hospital(高丽大学安岩医院);Seegene Medical Foundation(Seegene医学财团);Seoul National University(首尔大学);Ewha Womans University(梨花女子大学);The Catholic University of Korea(韩国天主教大学)
Interpretability and Individuality in Knee MRI: Patient-Specific Radiomic Fingerprint with Reconstructed Healthy Personas
organization= Department of Mechanical Engineering, University College London , city= London , country= UK;organization= Hawkes Institute, University College London , city= London , country= UK;organization= Institute of Orthopaedic \& Musculoskeletal Science, University College London, Royal National Orthopaedic Hospital , city= Stanmore , country= UK;organization= Royal National Orthopaedic Hospital , city= Stanmore , country= UK;organization= School of Engineering;Materials Science, Queen Mary University of London , city= London , country= UK;organization= Centre for Bioengineering, Queen Mary University of London , city= London , country= UK;organization= Department of Medical Physics;Biomedical Engineering, University College London , city= London , country= UK
PathFLIP: Fine-grained Language-Image Pretraining for Versatile Computational Pathology
Harbin Institute of Technology, Shenzhen, School of Computer Science and Technology(哈尔滨工业大学(深圳)计算机科学与技术学院);National University of Singapore, Department of Electronic and Computer Engineering(新加坡国立大学电子与计算机工程系)
Agentic Visual Reasoning in Whole-Slide Pathology Images via Active Perception
College of Computer Science and Technology, Harbin Institute of Technology, Shenzhen(哈尔滨工业大学(深圳)计算机科学与技术学院);School of Computing and Data Science, The University of Hong Kong(香港大学计算与数据科学学院)
PathAR: Structure-First Autoregressive Synthesis of Multimodal Pathology Images
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education(新一代人工智能技术及其交叉应用重点实验室(东南大学),教育部);Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, National University of Singapore(创新与精准眼健康中心,新加坡国立大学 Yong Loo Lin 医学院);Department of Ophthalmology, Yong Loo Lin School of Medicine, National University of Singapore(眼科学系,新加坡国立大学 Yong Loo Lin 医学院);Department of Biostatistics, Center for Global Health, School of Public Health, Nanjing Medical University(生物统计学系,全球健康中心,南京医科大学);Institute of High-Performance Computing, Agency for Science, Technology and Research(高性能计算研究所,科技研究局)
A Generative Foundation Model for Multimodal Histopathology
Department of Radiation Oncology, Stanford University School of Medicine(斯坦福大学医学院放射肿瘤学系);Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine(斯坦福大学医学院精神病学与行为科学系);Department of Statistics and Data Science, Yale University(耶鲁大学统计与数据科学系);Department of Computer Science, Stanford University(斯坦福大学计算机科学系);Department of Electrical Engineering, Stanford University(斯坦福大学电气工程系);Department of Biomedical Data Science, Stanford University(斯坦福大学生物医学数据科学系);Department of Radiology, Stanford University(斯坦福大学放射学系);Center for Artificial Intelligence in Medicine and Imaging, Stanford University(斯坦福大学医学与影像人工智能中心)
Self-Auditing Residual Drifting for Pathology-Preserving Accelerated Knee MRI
Department of Radiology & Biomedical Imaging, Yale School of Medicine(放射学与生物医学成像系,耶鲁医学院);Department of Biomedical Engineering, Rensselaer Polytechnic Institute(生物医学工程系,伦塞拉尔理工学院)
Diffusion-driven lensless fiber endomicroscopic quantitative phase imaging towards digital pathology
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University(同济大学上海智能自主系统研究院);Shanghai Artificial Intelligence Laboratory(上海人工智能实验室);Suzhou Institute of Biomedical Engineering and Technology, Chinese Academy of Sciences(中国科学院苏州生物医学工程技术研究所);School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University(西北工业大学人工智能与光电子学院);Institute of Artificial Intelligence (TeleAI), China Telecom Corp Ltd(中国电信股份有限公司人工智能研究院);Competence Center for Biomedical Computational Laser Systems (BIOLAS), TU Dresden(德累斯顿工业大学生物医学计算激光系统能力中心);Laboratory of Measurement and Sensor System Technique (MST), TU Dresden(德累斯顿工业大学测量与传感器系统技术实验室)
Thinking Like a Botanist: Challenging Multimodal Language Models with Intent-Driven Chain-of-Inquiry
Department of Robotics and Mechatronics Engineering, University of Dhaka(达卡大学机器人与机电工程系);Department of Computer Science and Engineering, University of Dhaka(达卡大学计算机科学与工程系);Department of Agronomy, Gazipur Agricultural University(加兹ipur农业大学作物学系);Department of Botany, University of Dhaka(达卡大学植物学系)
Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization
New York University Tandon School of Engineering(纽约大学坦登工程学院)
深度网络现在在大规模多中心成像研究中无处不在。然而,由于对比度、分辨率和噪声不一致,跨站点的图像直接聚合禁忌用于下游统计和基于深度学习的图像分析。为此,在没有配对数据的情况下,周期一致生成对抗网络的变体已被用于协调源域和目标域之间的图像集。重要的是,这些方法易于不稳定,对比度反转,棘手的病理操纵,和隐写映射,限制了它们在现实世界的医学成像中的可靠采用。在这项工作中,基于一个基本的假设,即形态形状是一致的成像部位,我们提出了一个分割重新规范化的图像翻译框架,以减少扫描仪间的异质性,同时保留解剖布局。我们用以共同学习的解剖分割嵌入为条件的可训练尺度和移位参数替换生成网络内标准化层中使用的仿射变换,以调节每个翻译级别的特征。我们在有和没有病变的数据集上,针对几种成像模式(T1 w MRI、FLAIR MRI和OCT)的最新基线评估了我们的方法。平移GANs的分割重正化产生了由Inception距离量化的卓越图像协调性,通过事后分割准确性证明了改进的下游效用,并提高了对平移扰动和自对抗攻击的鲁棒性。
Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis
Division of Computational Pathology, Department of Pathology and Laboratory Medicine, Indiana University School of Medicine(计算病理学部,病理学与实验室医学部,印第安纳大学医学院);IU Melvin and Bren Simon Comprehensive Cancer Center(印第安纳大学Melvin和Bren Simon综合癌症中心);Departments of Biostatistics and Health Data Science(生物统计学与健康数据科学部);Radiology and Imaging Sciences(放射学与影像科学部);Neurological Surgery(神经外科);Indiana University School of Medicine(印第安纳大学医学院);Department of Computer Science, Luddy School of Informatics, Computing, and Engineering(计算机科学部,Luddy信息、计算与工程学院)