Deep learning networks are being developed in every stage of the MRI workflow and have provided state-of-the-art results. However, this has come at the cost of increased computation requirement and storage. Hence, replacing the networks with compact models at various stages in the MRI workflow can significantly reduce the required storage space and provide considerable speedup. In computer vision, knowledge distillation is a commonly used method for model compression. In our work, we propose a knowledge distillation (KD) framework for the image to image problems in the MRI workflow in order to develop compact, low-parameter models without a significant drop in performance. We propose a combination of the attention-based feature distillation method and imitation loss and demonstrate its effectiveness on the popular MRI reconstruction architecture, DC-CNN. We conduct extensive experiments using Cardiac, Brain, and Knee MRI datasets for 4x, 5x and 8x accelerations. We observed that the student network trained with the assistance of the teacher using our proposed KD framework provided significant improvement over the student network trained without assistance across all the datasets and acceleration factors. Specifically, for the Knee dataset, the student network achieves $65\%$ parameter reduction, 2x faster CPU running time, and 1.5x faster GPU running time compared to the teacher. Furthermore, we compare our attention-based feature distillation method with other feature distillation methods. We also conduct an ablative study to understand the significance of attention-based distillation and imitation loss. We also extend our KD framework for MRI super-resolution and show encouraging results.
Revisiting Global Token Mixing in Task-Dependent MRI Restoration: Insights from Minimal Gated CNN Baselines
Dept. of Radiology & Imaging Sciences, University of Utah, Salt Lake City, UT, USA（放射学与成像科学系，犹他大学，盐湖城，UT，USA）；Dept. of Electrical & Computer Engineering, University of Utah, Salt Lake City, UT, USA（电气与计算机工程系，犹他大学，盐湖城，UT，USA）；Dept. of Computer Vision, Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE（计算机视觉系，莫扎德人工智能大学，阿布扎赫，EUA）；Dept. of Biomedical Engineering, University of Utah, Salt Lake City, UT, USA（生物医学工程系，犹他大学，盐湖城，UT，USA）；Dept. of Electrical & Computer Engineering, University of Washington, Seattle, WA, USA（电气与计算机工程系，华盛顿大学，西雅图，WA，USA）
Distribution-aware Dataset Distillation for Efficient Image Restoration
Sun Yat‑sen University（中山大学）；Fuzhou University（福州大学）；University of Science and Technology of China（中国科学技术大学）；Nanjing University of Science and Technology（南京理工大学）
Distillation-Driven Diffusion Model for Multi-Scale MRI Super-Resolution: Make 1.5T MRI Great Again
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；University Sorbonne Paris Nord（巴黎北索邦大学）；Jiangsu Institute of Hematology（江苏血液学研究所）；The First Affiliated Hospital of Soochow University（苏州大学附属第一医院）；Henan University of Chinese Medicine（河南中医药大学）；German Cancer Research Center（德国癌症研究中心）；Athinoula A. Martinos Centre for Biomedical Imaging（阿西努拉·A.马蒂诺斯生物医学成像中心）；Geneva University Hospital（日内瓦大学医院）；IDP Institute（IDP研究所）；University of Orleans（奥尔良大学）
Courant Institute of Mathematical Sciences（库朗数学科学研究所）；New York University（纽约大学）；Vilcek Institute of Graduate Biomedical Sciences（维尔切克生物医学研究生科学研究所）
Multi-modal magnetic resonance imaging (MRI) is a crucial method for analyzing the human brain. It is usually used for diagnosing diseases and for making valuable decisions regarding the treatments - for instance, checking for gliomas in the human brain. With varying degrees of severity and detection, properly diagnosing gliomas is one of the most daunting and significant analysis tasks in modern-day medicine. Our primary focus is on working with different approaches to perform the segmentation of brain tumors in multimodal MRI scans. Now, the quantity, variability of the data used for trainin…
Enriched text-guided variational multimodal knowledge distillation network (VMD) for automated diagnosis of plaque vulnerability in 3D carotid artery MRI
department of Radiology and Nuclear Medicine, Xuanwu Hospital, Capital Medical University（放射医学与核医学科，宣武医院，首都医科大学）；Beijing Key Laboratory of Magnetic Resonance Imaging and Brain Informatics（北京磁共振成像与脑信息学重点实验室）；Beijing United Imaging Research Institute of Intelligent Imaging（北京智能成像联合研究院）
McGill University（麦吉尔大学）；Centre for Intelligent Machines (CIM)（智能机器中心 (CIM)）；Department of Electrical and Computer Engineering（电气与计算机工程系）
Segmentation of enhancing tumours or lesions from MRI is important for detecting new disease activity in many clinical contexts. However, accurate segmentation requires the inclusion of medical images (e.g., T1 post contrast MRI) acquired after injecting patients with a contrast agent (e.g., Gadolinium), a process no longer thought to be safe. Although a number of modality-agnostic segmentation networks have been developed over the past few years, they have been met with limited success in the context of enhancing pathology segmentation. In this work, we present HAD-Net, a novel offline advers…
University of Science and Technology of China（中国科学技术大学）；Microsoft Research（微软研究院）；University of Rochester（罗切斯特大学）
An important scenario for image quality assessment (IQA) is to evaluate image restoration (IR) algorithms. The state-of-the-art approaches adopt a full-reference paradigm that compares restored images with their corresponding pristine-quality images. However, pristine-quality images are usually unavailable in blind image restoration tasks and real-world scenarios. In this paper, we propose a practical solution named degraded-reference IQA (DR-IQA), which exploits the inputs of IR models, degraded images, as references. Specifically, we extract reference information from degraded images by dist…
Lightweight Medical Image Restoration via Integrating Reliable Lesion-Semantic Driven Prior
University of Electronic Science and Technology of China（电子科技大学）；City University of Hong Kong（香港城市大学）；Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；Shenzhen Institute For Advanced Study, University of Electronic Science and Technology of China（电子科技大学深圳研究院）
Image restoration is the task of recovering a clean image from a degraded version. In most cases, the degradation is spatially varying, and it requires the restoration network to both localize and restore the affected regions. In this paper, we present a new approach suitable for handling the image-specific and spatially-varying nature of degradation in images affected by practically occurring artifacts such as blur, rain-streaks. We decompose the restoration task into two stages of degradation localization and degraded region-guided restoration, unlike existing methods which directly learn a …
We apply basic statistical reasoning to signal reconstruction by machine learning -- learning to map corrupted observations to clean signals -- with a simple and powerful conclusion: it is possible to learn to restore images by only looking at corrupted examples, at performance at and sometimes exceeding training using clean data, without explicit image priors or likelihood models of the corruption. In practice, we show that a single model learns photographic noise removal, denoising synthetic Monte Carlo images, and reconstruction of undersampled MRI scans -- all corrupted by different proces…
Medical University of Vienna（维也纳医科大学）；Intel（英特尔公司）；Lobachevsky State University（罗巴切夫斯基国立大学）
The segmentation of brain tumors in multimodal MRIs is one of the most challenging tasks in medical image analysis. The recent state of the art algorithms solving this task is based on machine learning approaches and deep learning in particular. The amount of data used for training such models and its variability is a keystone for building an algorithm with high representation power. In this paper, we study the relationship between the performance of the model and the amount of data employed during the training process. On the example of brain tumor segmentation challenge, we compare the model…
Boosting ViT-based MRI Reconstruction from the Perspectives of Frequency Modulation, Spatial Purification, and Scale Diversification
Fudan University（复旦大学）；Digital Medical Research Center（数字医学研究中心）；School of Basic Medical Science（基础医学院）；Shanghai Key Laboratory of Medical Image Computing and Computer Assisted Intervention（上海医学影像计算与计算机辅助干预重点实验室）；Academy for Engineering and Technology（工程与技术学院）
With the recently massive development in convolution neural networks, numerous lightweight CNN-based image super-resolution methods have been proposed for practical deployments on edge devices. However, most existing methods focus on one specific aspect: network or loss design, which leads to the difficulty of minimizing the model size. To address the issue, we conclude block devising, architecture searching, and loss design to obtain a more efficient SR structure. In this paper, we proposed an edge-enhanced feature distillation network, named EFDN, to preserve the high-frequency information u…
Distilling Missing Modality Knowledge from Ultrasound for Endometriosis Diagnosis with Magnetic Resonance Images
Australian Institute for Machine Learning, University of Adelaide（阿德莱德大学澳大利亚机器学习研究所）；Flinders Health and Medical Research Institute, Flinders University（弗林德斯大学弗林德斯健康与医学研究所）；Robinson Research Institute, University of Adelaide（阿德莱德大学罗宾逊研究所）；Centre for Vision, Speech and Signal Processing, University of Surrey（萨里大学视觉、语音与信号处理中心）
State-of-the-art algorithms for imaging inverse problems (namely deblurring and reconstruction) are typically iterative, involving a denoising operation as one of its steps. Using a state-of-the-art denoising method in this context is not trivial, and is the focus of current work. Recently, we have proposed to use a class-adapted denoiser (patch-based using Gaussian mixture models) in a so-called plug-and-play scheme, wherein a state-of-the-art denoiser is plugged into an iterative algorithm, leading to results that outperform the best general-purpose algorithms, when applied to an image of a …
All-in-One Medical Image Restoration with Latent Diffusion-Enhanced Vector-Quantized Codebook Prior
School of Biological Science and Medical Engineering（生物科学与医学工程学院）；State Key Laboratory of Software Development Environment（软件开发环境国家重点实验室）；Key Laboratory of Biomechanics and Mechanobiology of Ministry of Education（教育部生物力学与机械生物学重点实验室）；Beijing Advanced Innovation Center for Biomedical Engineering（北京生物医学工程先进创新中心）；Beihang University（北京航空航天大学）；Department of Biomedical Engineering（生物医学工程系）；Tsinghua University（清华大学）；ByteDance Inc.（字节跳动公司）
We propose an efficient neural network for RAW image denoising. Although neural network-based denoising has been extensively studied for image restoration, little attention has been given to efficient denoising for compute limited and power sensitive devices, such as smartphones and smartwatches. In this paper, we present a novel architecture and a suite of training techniques for high quality denoising in mobile devices. Our work is distinguished by three main contributions. (1) Feature-Align layer that modulates the activations of an encoder-decoder architecture with the input noisy images. …
Unsupervised lesion detection is a challenging problem that requires accurately estimating normative distributions of healthy anatomy and detecting lesions as outliers without training examples. Recently, this problem has received increased attention from the research community following the advances in unsupervised learning with deep learning. Such advances allow the estimation of high-dimensional distributions, such as normative distributions, with higher accuracy than previous methods.The main approach of the recently proposed methods is to learn a latent-variable model parameterized with n…
Echocardiography to Cardiac MRI View Transformation for Real-Time Blind Restoration
Tampere University（坦佩雷大学）；Qatar University（卡塔尔大学）；VTT Technical Research Centre of Finland（芬兰国家技术研究中心）；Hamad Medical Corporation Hospital（哈马德医疗公司医院）；Hayatabad Medical Complex（海亚特阿巴德医疗中心）
Compressed sensing (CS) is an efficient method to reconstruct MR image from small sampled data in $k$-space and accelerate the acquisition of MRI. In this work, we propose a novel deep geometric distillation network which combines the merits of model-based and deep learning-based CS-MRI methods, it can be theoretically guaranteed to improve geometric texture details of a linear reconstruction. Firstly, we unfold the model-based CS-MRI optimization problem into two sub-problems that consist of image linear approximation and image geometric compensation. Secondly, geometric compensation sub-prob…
Learning Efficient and Effective Trajectories for Differential Equation-based Image Restoration
Department of Computer Science, City University of Hong Kong（香港城市大学计算机科学系）；School of Computing and Information Sciences, Saint Francis University（圣芳济大学计算与信息科学学院）；School of Engineering, Huaqiao University（华侨大学工程学院）；School of Information Science and Engineering, Huaqiao University（华侨大学信息科学与工程学院）
TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration
School of Computer Science, Northwestern Polytechnical University, Xi'an, China（西北工业大学计算机学院，西安，中国）；Shenzhen Research Institute of Northwestern Polytechnical University, Shenzhen, China（西北工业大学深圳研究院，深圳，中国）
SSCM: A Spatial-Semantic Consistent Model for Multi-Contrast MRI Super-Resolution
University of Science and Technology of China（科学技术大学）；Anhui Province Key Laboratory of Biomedical Imaging and Intelligent Processing（安徽省生物医学成像与智能处理重点实验室）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（人工智能研究所）；Beijing Tiantan Hospital, Capital Medical University（北京天坛医院）
Memorial Sloan Kettering Cancer Center（纪念斯隆·凯特琳癌症中心）
Accurate and robust segmentation of lung cancers from CT, even those located close to mediastinum, is needed to more accurately plan and deliver radiotherapy and to measure treatment response. Therefore, we developed a new cross-modality educed distillation (CMEDL) approach, using unpaired CT and MRI scans, whereby an informative teacher MRI network guides a student CT network to extract features that signal the difference between foreground and background. Our contribution eliminates two requirements of distillation methods: (i) paired image sets by using an image to image (I2I) translation a…
How to properly model the inter-frame relation within the video sequence is an important but unsolved challenge for video restoration (VR). In this work, we propose an unsupervised flow-aligned sequence-to-sequence model (S2SVR) to address this problem. On the one hand, the sequence-to-sequence model, which has proven capable of sequence modeling in the field of natural language processing, is explored for the first time in VR. Optimized serialization modeling shows potential in capturing long-range dependencies among frames. On the other hand, we equip the sequence-to-sequence model with an u…
