Cascaded Multi-path Shortcut Diffusion Model for Medical Image Translation
Yale University（耶鲁大学）；University of California Irvine（加州大学欧文分校）；University of Science and Technology of China（中国科学技术大学）；Case Western Reserve University（凯斯西储大学）；Northwestern University（西北大学）；Yale School of Medicine（耶鲁医学院）
Yinchi Zhou, Tianqi Chen, Jun Hou, Huidong Xie, Nicha C. Dvornek, S. Kevin Zhou, David L. Wilson, James S. Duncan, Chi Liu, Bo Zhou
Image-to-image translation is a vital component in medical imaging processing, with many uses in a wide range of imaging modalities and clinical scenarios. Previous methods include Generative Adversarial Networks (GANs) and Diffusion Models (DMs), which offer realism but suffer from instability and lack uncertainty estimation. Even though both GAN and DM methods have individually exhibited their capability in medical image translation tasks, the potential of combining a GAN and DM to further improve translation performance and to enable uncertainty estimation remains largely unexplored. In this work, we address these challenges by proposing a Cascade Multi-path Shortcut Diffusion Model (CMDM) for high-quality medical image translation and uncertainty estimation. To reduce the required number of iterations and ensure robust performance, our method first obtains a conditional GAN-generated prior image that will be used for the efficient reverse translation with a DM in the subsequent step. Additionally, a multi-path shortcut diffusion strategy is employed to refine translation results and estimate uncertainty. A cascaded pipeline further enhances translation quality, incorporating residual averaging between cascades. We collected three different medical image datasets with two sub-tasks for each dataset to test the generalizability of our approach. Our experimental results found that CMDM can produce high-quality translations comparable to state-of-the-art methods while providing reasonable uncertainty estimations that correlate well with the translation error.
LMU Munich（慕尼黑大学）；University Hospital, LMU Munich（慕尼黑大学附属医院）；Munich Center for Machine Learning（慕尼黑机器学习中心）
While recent advances in large-scale foundational models show promising results, their application to the medical domain has not yet been explored in detail. In this paper, we progress into the realms of large-scale modeling in medical synthesis by proposing Cheff - a foundational cascaded latent diffusion model, which generates highly-realistic chest radiographs providing state-of-the-art quality on a 1-megapixel scale. We further propose MaCheX, which is a unified interface for public chest datasets and forms the largest open collection of chest X-rays up to date. With Cheff conditioned on r…
Memory-efficient High-resolution OCT Volume Synthesis with Cascaded Amortized Latent Diffusion Models
Nanjing University of Science and Technology（南京理工大学）；Institute of High Performance Computing, A*STAR（新加坡科技研究局高性能计算研究所）；The Chinese University of Hong Kong（香港中文大学）；The First Affiliated Hospital with Nanjing Medical University（南京医科大学第一附属医院）
Objective : Abdominal anatomy segmentation is crucial for numerous applications from computer-assisted diagnosis to image-guided surgery. In this context, we address fully-automated multi-organ segmentation from abdominal CT and MR images using deep learning. Methods: The proposed model extends standard conditional generative adversarial networks. Additionally to the discriminator which enforces the model to create realistic organ delineations, it embeds cascaded partially pre-trained convolutional encoder-decoders as generator. Encoder fine-tuning from a large amount of non-medical images all…
Technical University of Denmark（丹麦技术大学）；Slagelse Hospital（斯劳厄尔瑟医院）；CAMES；Copenhagen University Hospital Rigshospitalet（哥本哈根大学医院 Rigshospitalet）；University of Copenhagen（哥本哈根大学）
The superior performance of CNN on medical image analysis heavily depends on the annotation quality, such as the number of labeled image, the source of image, and the expert experience. The annotation requires great expertise and labour. To deal with the high inter-rater variability, the study of imperfect label has great significance in medical image segmentation tasks. In this paper, we present a novel cascaded robust learning framework for chest X-ray segmentation with imperfect annotation. Our model consists of three independent network, which can effectively learn useful information from …
Intracranial tumors are groups of cells that usually grow uncontrollably. One out of four cancer deaths is due to brain tumors. Early detection and evaluation of brain tumors is an essential preventive medical step that is performed by magnetic resonance imaging (MRI). Many segmentation techniques exist for this purpose. Low segmentation accuracy is the main drawback of existing methods. In this paper, we use a deep learning method to boost the accuracy of tumor segmentation in MR images. Cascade approach is used with multiple scales of images to induce both local and global views and help the…
Umeå University（于默奥大学）；Eindhoven University of Technology（埃因霍温理工大学）
Kidney tumor segmentation emerges as a new frontier of computer vision in medical imaging. This is partly due to its challenging manual annotation and great medical impact. Within the scope of the Kidney Tumor Segmentation Challenge 2019, that is aiming at combined kidney and tumor segmentation, this work proposes a novel combination of 3D U-Nets---collectively denoted TuNet---utilizing the resulting kidney masks for the consecutive tumor segmentation. The proposed method achieves a Sørensen-Dice coefficient score of 0.902 for the kidney, and 0.408 for the tumor segmentation, computed from a f…
Hybrid imaging promises large potential in medical imaging applications. To fully utilize the possibilities of corresponding information from different modalities, the information must be transferable between the domains. In radiation therapy planning, existing methods make use of reconstructed 3D magnetic resonance imaging data to synthesize corresponding X-ray attenuation maps. In contrast, for fluoroscopic procedures only line integral data, i.e., 2D projection images, are present. The question arises which approaches could potentially be used for this MR to X-ray projection image-to-image …
Cascaded Multi-Modal Mixing Transformers for Alzheimer's Disease Classification with Incomplete Data
Queensland Brain Institute, The University of Queensland（昆士兰脑研究所，昆士兰大学）；School of Information Technology and Electrical Engineering, The University of Queensland（信息技术与电气工程学院，昆士兰大学）
Adaptive Domain Shift in Diffusion Models for Cross-Modality Image Translation
Laplace Lab at University of Tennessee（田纳西大学Laplace实验室）；University of California Riverside（加州大学河滨分校）；University of Tennessee at Chattanooga（田纳西大学查塔努加分校）
Safeguarding Medical Image Segmentation Datasets against Unauthorized Training via Contour- and Texture-Aware Perturbations
Beihang University（北京航空航天大学）；Nanyang Technological University（南洋理工大学）；Memorial Sloan Kettering Cancer Center（纪念斯隆凯特琳癌症中心）；Great Bay University（大湾区大学）；Beijing Institute of Technology（北京理工大学）
The accurate segmentation of multiple types of lesions from adjacent tissues in medical images is significant in clinical practice. Convolutional neural networks (CNNs) based on the coarse-to-fine strategy have been widely used in this field. However, multi-lesion segmentation remains to be challenging due to the uncertainty in size, contrast, and high interclass similarity of tissues. In addition, the commonly adopted cascaded strategy is rather demanding in terms of hardware, which limits the potential of clinical deployment. To address the problems above,we propose a novel Prior Attention N…
Bayesian generative models can flag performance loss, bias, and out-of-distribution image content
Instituto Universitario de Investigación en Tecnología Centrada en el Ser Humano, Universitat Politècnica de València（瓦伦西亚理工大学人类中心技术研究大学学院）；Technical University of Denmark（丹麦技术大学）
In the Picture: Medical Imaging Datasets, Artifacts, and their Living Review
Radboudumc Netherlands；KU \& Rigshospitalet Denmark；Emory University USA；Stanford University USA；RUG Netherlands；Radboudumc \& Plain Medical Netherlands；Oxford University Hospitals UK；Copenhagen University Hospital \& RAIT Denmark；KU \& Rigshospitalet；Emory University；Stanford University；Radboudumc \& Plain Medical；Oxford University Hospitals；Copenhagen University Hospital \& RAIT；Institute for Clarity in Documentation P.O. Box 1212 Dublin Ohio USA 43017-6221；Inria Paris-Rocquencourt Rocquencourt France；Rajiv Gandhi University Rono-Hills Doimukh Arunachal Pradesh India；Tsinghua University 30 Shuangqing Rd Haidian Qu Beijing Shi China；Palmer Research Laboratories 8600 Datapoint Drive San Antonio Texas USA 78229；Institute for Clarity in Documentation；Inria Paris-Rocquencourt；Rajiv Gandhi University；Tsinghua University；Palmer Research Laboratories；Institute for Clarity in Documentation Dublin Ohio USA
Multi-scale Cascaded Foundation Model for Whole-body Organs-at-risk Segmentation
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology（人工智能与自动化学院，华中科技大学）；Institute of Artificial Intelligence, Huazhong University of Science and Technology（人工智能研究院，华中科技大学）；Hubei Key Laboratory of Brain-Inspired Intelligent Systems, Huazhong University of Science and Technology（湖北省脑启发智能系统重点实验室，华中科技大学）；Key Laboratory of Image Processing and Intelligent Control (Huazhong University of Science and Technology), Ministry of Education（图像处理与智能控制重点实验室（华中科技大学），教育部）；Key Laboratory of Intelligent Computing and Signal Processing, Ministry of Education, Anhui University（智能计算与信号处理重点实验室（安徽大学），教育部）；College of Computing and Data Science (CCDS), Nanyang Technological University（计算与数据科学学院（CCDS），南洋理工大学）；East China University of Science and Technology（东华大学）
A Symmetric Dynamic Learning Framework for Diffeomorphic Medical Image Registration
Xiangtan University（湘潭大学）；University of Strathclyde（思克莱德大学）；Nankai University（南开大学）；Academy of Mathematics and Systems Science, Chinese Academy of Sciences（中国科学院数学与系统科学研究院）；University of Manchester（曼彻斯特大学）
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Science and Technology of China（中国科学技术大学）
Medical image segmentation can provide a reliable basis for further clinical analysis and disease diagnosis. The performance of medical image segmentation has been significantly advanced with the convolutional neural networks (CNNs). However, most existing CNNs-based methods often produce unsatisfactory segmentation mask without accurate object boundaries. This is caused by the limited context information and inadequate discriminative feature maps after consecutive pooling and convolution operations. In that the medical image is characterized by the high intra-class variation, inter-class indi…
Xinjiang University（新疆大学）；Central South University（中南大学）
Medical image classification has developed rapidly under the impetus of the convolutional neural network (CNN). Due to the fixed size of the receptive field of the convolution kernel, it is difficult to capture the global features of medical images. Although the self-attention-based Transformer can model long-range dependencies, it has high computational complexity and lacks local inductive bias. Much research has demonstrated that global and local features are crucial for image classification. However, medical images have a lot of noisy, scattered features, intra-class variation, and inter-cl…
Skull-stripping induces shortcut learning in MRI-based Alzheimer's disease classification
Department of Neurology, Medical University of Graz（格拉茨医科大学神经科）；Institute of Biomedical Imaging, Graz University of Technology（格拉茨技术大学生物医学成像研究所）；BioTechMed-Graz（格拉茨BioTechMed）
Institute for Parallel and Distributed Systems, University of Stuttgart（斯图加特大学并行与分布式系统研究所）
Image registration is a fundamental task for medical imaging. Resampling of the intensity values is required during registration and better spatial resolution with finer and sharper structures can improve the resampling performance and hence the registration accuracy. Super-resolution (SR) is an algorithmic technique targeting at spatial resolution enhancement which can achieve an image resolution beyond the hardware limitation. In this work, we consider SR as a preprocessing technique and present a CNN-based resolution enhancement module (REM) which can be easily plugged into the registration…
Mobile U-ViT: Revisiting large kernel and U-shaped ViT for efficient medical image segmentation
University of Science and Technology of China（中国科学技术大学）；Suzhou Institute for Advanced Research, USTC（苏州先进研究院）；Shanghai Jiao Tong University（上海交通大学）；Harbin Institute of Technology（哈尔滨工业大学）；State Grid Hunan ElectricPower Corporation Limited Research Institute（国网湖南电力有限公司研究院）
HER-Seg: Holistically Efficient Segmentation for High-Resolution Medical Images
School of Computer Science, University of Nottingham Ningbo China（诺丁汉大学宁波校区计算机科学学院）；University of Nottingham, UK（诺丁汉大学（英国））；Sichuan University Pittsburgh Institute, Sichuan University, China（四川大学匹兹堡研究院，四川大学，中国）；The Chinese University of Hong Kong, Hong Kong SAR（香港中文大学（香港特别行政区））；School of Computer Science, University of Nottingham, UK（诺丁汉大学（英国）计算机科学学院）；School of Computer Science, University of Lincoln, UK（林肯大学（英国）计算机科学学院）；HKISI, Chinese Academy of Sciences, Hong Kong SAR（香港科学院HKISI，香港特别行政区）
The Hong Kong Polytechnic University（香港理工大学）；University of Oxford（牛津大学）
Automatic medical image segmentation based on Computed Tomography (CT) has been widely applied for computer-aided surgery as a prerequisite. With the development of deep learning technologies, deep convolutional neural networks (DCNNs) have shown robust performance in automated semantic segmentation of medical images. However, semantic segmentation algorithms based on DCNNs still meet the challenges of feature loss between encoder and decoder, multi-scale object, restricted field of view of filters, and lack of medical image data. This paper proposes a novel algorithm for automated vertebrae s…
EG-SpikeFormer: Eye-Gaze Guided Transformer on Spiking Neural Networks for Medical Image Analysis
School of Computing, University of Georgia（佐治亚大学计算学院）；University of Texas at Arlington（德克萨斯大学阿灵顿分校）；Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；University of Georgia（佐治亚大学）；Department of Physics and Astronomy（物理与天文学系）
With substantial public concerns on potential cancer risks and health hazards caused by the accumulated radiation exposure in medical imaging, reducing radiation dose in X-ray based medical imaging such as Computed Tomography Perfusion (CTP) has raised significant research interests. In this paper, we embrace the deep Convolutional Neural Networks (CNN) based approaches and introduce Smoothed Dense-Convolution Neural Network (SDCNet) to recover high-dose quality CTP images from low-dose ones. SDCNet is composed of sub-network blocks cascaded by skip-connections to infer the noise (differential…
Hefei University of Technology（合肥工业大学）；Harbin Institute of Technology (Shenzhen)（哈尔滨工业大学（深圳））
Single image deraining task is still a very challenging task due to its ill-posed nature in reality. Recently, researchers have tried to fix this issue by training the CNN-based end-to-end models, but they still cannot extract the negative rain streaks from rainy images precisely, which usually leads to an over de-rained or under de-rained result. To handle this issue, this paper proposes a new coarse-to-fine single image deraining framework termed Multi-stream Hybrid Deraining Network (shortly, MH-DerainNet). To obtain the negative rain streaks during training process more accurately, we pres…
Charité - Universitätsmedizin Berlin（柏林夏里特医学院）；University of Innsbruck（因斯布鲁克大学）；Physikalisch-Technische Bundesanstalt (PTB)（德国联邦物理技术研究院）；Technical University of Berlin（柏林工业大学）；German Cancer Research Center（德国癌症研究中心）
In this paper we present a generalized Deep Learning-based approach for solving ill-posed large-scale inverse problems occuring in medical image reconstruction. Recently, Deep Learning methods using iterative neural networks and cascaded neural networks have been reported to achieve state-of-the-art results with respect to various quantitative quality measures as PSNR, NRMSE and SSIM across different imaging modalities. However, the fact that these approaches employ the forward and adjoint operators repeatedly in the network architecture requires the network to process the whole images or volu…
The pandemic of novel SARS-CoV-2 also known as COVID-19 has been spreading worldwide, causing rampant loss of lives. Medical imaging such as CT, X-ray, etc., plays a significant role in diagnosing the patients by presenting the visual representation of the functioning of the organs. However, for any radiologist analyzing such scans is a tedious and time-consuming task. The emerging deep learning technologies have displayed its strength in analyzing such scans to aid in the faster diagnosis of the diseases and viruses such as COVID-19. In the present article, an automated deep learning based mo…
