Preservational Learning Improves Self-supervised Medical Image Models by Reconstructing Diverse Contexts
The University of Hong Kong（香港大学）；ShanghaiTech University（上海科技大学）；Deepwise AI Lab（深睿人工智能实验室）；Shenzhen Research Institute of Big Data, The Chinese University of Hong Kong (Shenzhen)（香港中文大学（深圳）深圳大数据研究院）
Preserving maximal information is one of principles of designing self-supervised learning methodologies. To reach this goal, contrastive learning adopts an implicit way which is contrasting image pairs. However, we believe it is not fully optimal to simply use the contrastive estimation for preservation. Moreover, it is necessary and complemental to introduce an explicit solution to preserve more information. From this perspective, we introduce Preservational Learning to reconstruct diverse image contexts in order to preserve more information in learned representations. Together with the contrastive loss, we present Preservational Contrastive Representation Learning (PCRL) for learning self-supervised medical representations. PCRL provides very competitive results under the pretraining-finetuning protocol, outperforming both self-supervised and supervised counterparts in 5 classification/segmentation tasks substantially.
PRIOR: Prototype Representation Joint Learning from Medical Images and Reports
Southern University of Science and Technology（南方科技大学）；Jiaxing Research Institute, Southern University of Science and Technology（南方科技大学嘉兴研究院）；The University of Hong Kong（香港大学）；University of British Columbia（不列颠哥伦比亚大学）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）
Shape priors have long been known to be effective when reconstructing 3D shapes from noisy or incomplete data. When using a deep-learning based shape representation, this often involves learning a latent representation, which can be either in the form of a single global vector or of multiple local ones. The latter allows more flexibility but is prone to overfitting. In this paper, we advocate a hybrid approach representing shapes in terms of 3D meshes with a separate latent vector at each vertex. During training the latent vectors are constrained to have the same value, which avoids overfittin…
The Athenian School（雅典纳中学）；Cornell University（康奈尔大学）
Because of the necessity to obtain high-quality images with minimal radiation doses, such as in low-field magnetic resonance imaging, super-resolution reconstruction in medical imaging has become more popular (MRI). However, due to the complexity and high aesthetic requirements of medical imaging, image super-resolution reconstruction remains a difficult challenge. In this paper, we offer a deep learning-based strategy for reconstructing medical images from low resolutions utilizing Transformer and Generative Adversarial Networks (T-GAN). The integrated system can extract more precise texture …
Image restoration is a typical ill-posed problem, and it contains various tasks. In the medical imaging field, an ill-posed image interrupts diagnosis and even following image processing. Both traditional iterative and up-to-date deep networks have attracted much attention and obtained a significant improvement in reconstructing satisfying images. This study combines their advantages into one unified mathematical model and proposes a general image restoration strategy to deal with such problems. This strategy consists of two modules. First, a novel generative adversarial net(GAN) with WGAN-GP …
Reconstructing medical images from partial measurements is an important inverse problem in Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). Existing solutions based on machine learning typically train a model to directly map measurements to medical images, leveraging a training dataset of paired images and measurements. These measurements are typically synthesized from images using a fixed physical model of the measurement process, which hinders the generalization capability of models to unknown measurement processes. To address this issue, we propose a fully unsupervised techniq…
Shenzhen Research Institute of Big Data（深圳大数据研究院）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；Sun Yat-sen University（中山大学）；Pazhou Lab（琶洲实验室）
Medical vision-and-language pre-training provides a feasible solution to extract effective vision-and-language representations from medical images and texts. However, few studies have been dedicated to this field to facilitate medical vision-and-language understanding. In this paper, we propose a self-supervised learning paradigm with multi-modal masked autoencoders (M$^3$AE), which learn cross-modal domain knowledge by reconstructing missing pixels and tokens from randomly masked images and texts. There are three key designs to make this simple approach work. First, considering the different …
X-ray computed tomography (CT) reconstructs cross-sectional images from projection data. However, ionizing X-ray radiation associated with CT scanning might induce cancer and genetic damage. Therefore, the reduction of radiation dose has attracted major attention. Few-view CT image reconstruction is an important topic to reduce the radiation dose. Recently, data-driven algorithms have shown great potential to solve the few-view CT problem. In this paper, we develop a dual network architecture (DNA) for reconstructing images directly from sinograms. In the proposed DNA method, a point-based ful…
Durham University（杜伦大学）；County Durham and Darlington NHS Foundation Trust（达勒姆郡和达灵顿国民保健服务基金会信托）
Computed tomography (CT) is an effective medical imaging modality, widely used in the field of clinical medicine for the diagnosis of various pathologies. Advances in Multidetector CT imaging technology have enabled additional functionalities, including generation of thin slice multiplanar cross-sectional body imaging and 3D reconstructions. However, this involves patients being exposed to a considerable dose of ionising radiation. Excessive ionising radiation can lead to deterministic and harmful effects on the body. This paper proposes a Deep Learning model that learns to reconstruct CT proj…
FaRMamba: Frequency-based learning and Reconstruction aided Mamba for Medical Segmentation
School of Artificial Intelligence and Computer Science, Nantong University, China（人工智能与计算机科学学院，南通大学）；School of Information Science and Technology, Nantong University, China（信息科学与技术学院，南通大学）
Anonymization of medical images is necessary for protecting the identity of the test subjects, and is therefore an essential step in data sharing. However, recent developments in deep learning may raise the bar on the amount of distortion that needs to be applied to guarantee anonymity. To test such possibilities, we have applied the novel CycleGAN unsupervised image-to-image translation framework on sagittal slices of T1 MR images, in order to reconstruct facial features from anonymized data. We applied the CycleGAN framework on both face-blurred and face-removed images. Our results show that…
Boosting Vision Semantic Density with Anatomy Normality Modeling for Medical Vision-language Pre-training
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；DAMO Academy, Alibaba Group（阿里巴巴集团达摩院）；The First Affiliated Hospital of College of Medicine, Zhejiang University（浙江大学医学院附属第一医院）；Hupan Lab（虎盘实验室）
The Chinese University of Hong Kong（香港中文大学）；Imsight Medical Technology Inc（宜远智能医疗科技有限公司）；Xiamen University（厦门大学）；The Hong Kong Polytechnic University（香港理工大学）
Lymph node metastasis is one of the most significant diagnostic indicators in breast cancer, which is traditionally observed under the microscope by pathologists. In recent years, computerized histology diagnosis has become one of the most rapidly expanding fields in medical image computing, which alleviates pathologists' workload and reduces misdiagnosis rate. However, automatic detection of lymph node metastases from whole slide images remains a challenging problem, due to the large-scale data with enormous resolutions and existence of hard mimics. In this paper, we propose a novel framework…
Huazhong University of Science and Technology（华中科技大学）；Tongji Hospital（同济医院）
High-resolution 3D medical images are important for analysis and diagnosis, but axial scanning to acquire them is very time-consuming. In this paper, we propose a fast end-to-end multi-focal plane imaging network (MFPINet) to reconstruct high-resolution multi-focal plane images from a single 2D low-resolution wild filed image without relying on scanning. To acquire realistic MFP images fast, the proposed MFPINet adopts generative adversarial network framework and the strategies of post-sampling and refocusing all focal planes at one time. We conduct a series experiments on cytology microscopy …
Cyclic 2.5D Perceptual Loss for Cross-Modal 3D Medical Image Synthesis: T1w MRI to Tau PET
Department of Artificial Intelligence, Hanyang University, Seoul, South Korea（人工智能系，翰阳大学，首尔，韩国）；Department of Electronic Engineering, Hanyang University, Seoul, South Korea（电子工程系，翰阳大学，首尔，韩国）；Division of Computer Engineering, Hankuk University of Foreign Studies, Yongin, South Korea（计算机工程系，韩国外语大学， Yongin，韩国）；Division of AI Data Convergence, Hankuk University of Foreign Studies, Yongin, South Korea（AI数据融合系，韩国外语大学， Yongin，韩国）；Division of Language & AI, Hankuk University of Foreign Studies, Seoul, South Korea（语言与AI系，韩国外语大学，首尔，韩国）
DICE: Diffusion Consensus Equilibrium for Sparse-view CT Reconstruction
Department of Systems and Informatics Engineering（系统与信息工程系）；Department of Electrical, Electronics, and Telecommunications Engineering（电气、电子与电信工程系）；Department of Physics（物理系）；Universidad Industrial de Santander（圣安德烈斯工业大学）
impuTMAE: Multi-modal Transformer with Masked Pre-training for Missing Modalities Imputation in Cancer Survival Prediction
Center for Applied AI, Skolkovo Institute of Science and Technology, Moscow, Russian Federation（应用人工智能中心，斯克尔科夫科学与技术研究所，莫斯科，俄罗斯联邦）；BIMAI-Lab, Biomedically Informed Artificial Intelligence Laboratory, University of Sharjah, Sharjah, United Arab Emirates（BIMAI实验室，生物医学导向人工智能实验室，肖贾联合大学，肖贾，阿拉伯联合酋长国）；Ivannikov Institute for System Programming of the Russian Academy of Sciences, Moscow, Russian Federation（伊万诺夫俄罗斯科学院系统编程研究所，莫斯科，俄罗斯联邦）
Shape of my heart: Cardiac models through learned signed distance functions
Institute for Applied Mathematics, University of Bonn（波恩大学应用数学研究所）；University of Graz（格拉茨大学）；Institute for Biological and Medical Engineering, Pontificia Universidad Católica de Chile（智利天主教大学生物与医学工程研究所）；Center for Computational Medicine in Cardiology, Università della Svizzera italiana（瑞士意大利语区大学心血管计算医学中心）；FernUni Schweiz（瑞士远程教育大学）；Instituto Cardiocentro Ticino, EOC（提契诺心脏中心研究所（EOC））；University of Trento（特伦托大学）
Huaqiao University（华侨大学）；Xidian University（西安电子科技大学）；Hangzhou Institute of Technology, Xidian University（西安电子科技大学杭州研究院）
Reconstruction of image by using convolutional neural networks (CNNs) has been vigorously studied in the last decade. Until now, there have being developed several techniques for imaging of a single object through scattering medium by using neural networks, however how to reconstruct images of more than one object simultaneously seems hard to realize. In this paper, we demonstrate an approach by using generative adversarial network (GAN) to reconstruct images of two adjacent objects through scattering media. We construct an imaging system for imaging of two adjacent objects behind the scatteri…
Med-K2N: Flexible K-to-N Modality Translation for Medical Image Synthesis
University of Science and Technology of China（中国科学技术大学）；Suzhou Institute of Biomedical Engineering and Technology（苏州生物医学工程与技术研究所）；Chinese Academy of Sciences（中国科学院）；The Third Affiliated Hospital of Sun Yat-sen University（中山大学第三附属医院）
ETH Zurich（苏黎世联邦理工学院）；Huawei Noah’s Ark Lab（华为诺亚方舟实验室）；Imperial College London（帝国理工学院）；King’s College London（伦敦国王学院）；Microsoft Mixed Reality and AI lab（微软混合现实与人工智能实验室）
Surface reconstruction from magnetic resonance (MR) imaging data is indispensable in medical image analysis and clinical research. A reliable and effective reconstruction tool should: be fast in prediction of accurate well localised and high resolution models, evaluate prediction uncertainty, work with as little input data as possible. Current deep learning state of the art (SOTA) 3D reconstruction methods, however, often only produce shapes of limited variability positioned in a canonical position or lack uncertainty evaluation. In this paper, we present a novel probabilistic deep learning ap…
School of Biomedical Engineering, Shanghai Jiao Tong University（上海交通大学生物医学工程学院）；School of Information Science and Technology, ShanghaiTech University（上海交通大学信息科学与技术学院）；School of Biomedical Engineering and the National Engineering Research Center of Advanced Magnetic Resonance Technologies for Diagnosis and Therapy (NERCAMRT), Shanghai Jiao Tong University（上海交通大学生物医学工程学院和先进磁共振诊断与治疗国家工程研究中心）
HistoSpeckle-Net: Mutual Information-Guided Deep Learning for high-fidelity reconstruction of complex OrganAMNIST images via perturbed Multimode Fibers
Department of Electrical Engineering, Syed Babar Ali School of Science and Engineering, Lahore University of Management Sciences（电气工程系，Syed Babar Ali科学与工程学院，拉合尔管理科学大学）
Generative deep learning for foundational video translation in ultrasound
Department of Medicine, Division of Cardiology（医学系，心脏病科）；Bakar Computational Health Sciences Institute（Bakar计算健康科学研究所）；University of California, San Francisco（加州大学旧金山分校）；UCSF-UC Berkeley Joint Program in Computational Precision Health（UCSF-伯克利计算精准健康联合项目）；Department of Radiology, Center for Intelligent Imaging（放射科，智能成像中心）
StructDiff: Structure-aware Diffusion Model for 3D Fine-grained Medical Image Synthesis
Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications (Southeast University), Ministry of Education, China（新一代人工智能技术及跨学科应用关键实验室（东南大学），中华人民共和国教育部，中国）；Jiangsu Province Joint International Research Laboratory of Medical Information Processing, Southeast University, Nanjing, China（江苏省医疗信息处理联合国际研究实验室（东南大学），南京市，中国）；Univ Rennes, CHU Rennes, Inserm, LTSI– UMR 1099, F-35000 Rennes, France（里昂大学，里昂大学医院，Inserm，LTSI– UMR 1099，法国里昂，法国）；Shanghai AI Laboratory（上海人工智能实验室）；School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China（计算机科学与技术学院，北京理工大学，北京，中国）；Department of Radiology, Jinling Hospital, Affiliated Hospital of Medical School, Nanjing University, Nanjing, China（放射科，南京大学附属医院，南京大学，南京，中国）
Tackling Structural Hallucination in Image Translation with Local Diffusion
University College London（伦敦大学学院）；AstraZeneca（阿斯利康）；Centre for Medical Image Computing（医学影像计算中心）；Department of Medical Physics and Biomedical Engineering, University College London（伦敦大学学院医学物理与生物医学工程系）；Department of Computer Science, University College London（伦敦大学学院计算机科学系）；Centre for AI, DS&AI, AstraZeneca, UK（阿斯利康AI、DS&AI中心（英国））
