Time-resolved MR angiography (tMRA) has been widely used for dynamic contrast enhanced MRI (DCE-MRI) due to its highly accelerated acquisition. In tMRA, the periphery of the k-space data are sparsely sampled so that neighbouring frames can be merged to construct one temporal frame. However, this view-sharing scheme fundamentally limits the temporal resolution, and it is not possible to change the view-sharing number to achieve different spatio-temporal resolution trade-off. Although many deep learning approaches have been recently proposed for MR reconstruction from sparse samples, the existing approaches usually require matched fully sampled k-space reference data for supervised training, which is not suitable for tMRA. This is because high spatio-temporal resolution ground-truth images are not available for tMRA. To address this problem, here we propose a novel unsupervised deep learning using optimal transport driven cycle-consistent generative adversarial network (cycleGAN). In contrast to the conventional cycleGAN with two pairs of generator and discriminator, the new architecture requires just a single pair of generator and discriminator, which makes the training much simpler and improves the performance. Reconstruction results using in vivo tMRA data set confirm that the proposed method can immediately generate high quality reconstruction results at various choices of view-sharing numbers, allowing us to exploit better trade-off between spatial and temporal resolution in time-resolved MR angiography.
Fudan University（复旦大学）；Shanghai East Hospital, Tongji University（同济大学附属东方医院）；Ruijin Hospital, Shanghai Jiao Tong University School of Medicine（上海交通大学医学院附属瑞金医院）
We proposed a novel unsupervised methodology named Disarranged Zone Learning (DZL) to automatically recognize stenosis in coronary angiography. The methodology firstly disarranges the frames in a video, secondly it generates an effective zone and lastly trains an encoder-decoder GRU model to learn the capability to recover disarranged frames. The breakthrough of our study is to discover and validate the Sequence Intensity (Recover Difficulty) is a measure of Coronary Artery Stenosis Status. Hence, the prediction accuracy of DZL is used as an approximator of coronary stenosis indicator. DZL is …
X-GAN: A Generative AI-Powered Unsupervised Model for Main Vessel Segmentation of Glaucoma Screening
Southern Methodist University（南方 Methodist 大学）；National Aeronautics and Space Administration（美国国家航空航天局）；University of Texas Southwestern Medical Center（德克萨斯西南医学中心）；Northeastern University（东北大学）
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）
Deformable image registration is one of the fundamental tasks in medical imaging. Classical registration algorithms usually require a high computational cost for iterative optimizations. Although deep-learning-based methods have been developed for fast image registration, it is still challenging to obtain realistic continuous deformations from a moving image to a fixed image with less topological folding problem. To address this, here we present a novel diffusion-model-based image registration method, called DiffuseMorph. DiffuseMorph not only generates synthetic deformed images through revers…
Institute of Computer Science and Technology, Peking University（北京大学计算机科学技术研究所）
Cross-modal hashing aims to map heterogeneous multimedia data into a common Hamming space, which can realize fast and flexible retrieval across different modalities. Unsupervised cross-modal hashing is more flexible and applicable than supervised methods, since no intensive labeling work is involved. However, existing unsupervised methods learn hashing functions by preserving inter and intra correlations, while ignoring the underlying manifold structure across different modalities, which is extremely helpful to capture meaningful nearest neighbors of different modalities for cross-modal retrie…
Image Sciences Institute, University Medical Center Utrecht（乌得勒支大学医学中心影像科学研究所）；University Medical Center Utrecht（乌得勒支大学医学中心）；Amsterdam University Medical Center（阿姆斯特丹大学医学中心）；Utrecht University（乌得勒支大学）
In patients with obstructive coronary artery disease, the functional significance of a coronary artery stenosis needs to be determined to guide treatment. This is typically established through fractional flow reserve (FFR) measurement, performed during invasive coronary angiography (ICA). We present a method for automatic and non-invasive detection of patients requiring ICA, employing deep unsupervised analysis of complete coronary arteries in cardiac CT angiography (CCTA) images. We retrospectively collected CCTA scans of 187 patients, 137 of them underwent invasive FFR measurement in 192 dif…
Large intra-class variation is the result of changes in multiple object characteristics. Images, however, only show the superposition of different variable factors such as appearance or shape. Therefore, learning to disentangle and represent these different characteristics poses a great challenge, especially in the unsupervised case. Moreover, large object articulation calls for a flexible part-based model. We present an unsupervised approach for disentangling appearance and shape by learning parts consistently over all instances of a category. Our model for learning an object representation i…
Institute of Information Science, Academia Sinica（中央研究院信息科学研究所）；National Tsing Hua University（国立清华大学）
This paper presents a "learning to learn" approach to figure-ground image segmentation. By exploring webly-abundant images of specific visual effects, our method can effectively learn the visual-effect internal representations in an unsupervised manner and uses this knowledge to differentiate the figure from the ground in an image. Specifically, we formulate the meta-learning process as a compositional image editing task that learns to imitate a certain visual effect and derive the corresponding internal representation. Such a generative process can help instantiate the underlying figure-groun…
Pretrained Reversible Generation as Unsupervised Visual Representation Learning
Xi’an Jiaotong University（西安交通大学）；Shanghai AI Laboratory（上海人工智能实验室）；SenseTime（商汤科技）；The Chinese University of Hong Kong（香港中文大学）；Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
University of Nottingham（诺丁汉大学）；Samsung AI Center Cambridge（三星剑桥人工智能中心）；Hartpury University（哈特伯瑞大学）；Queen Mary University of London（伦敦玛丽女王大学）
This paper proposes a novel paradigm for the unsupervised learning of object landmark detectors. Contrary to existing methods that build on auxiliary tasks such as image generation or equivariance, we propose a self-training approach where, departing from generic keypoints, a landmark detector and descriptor is trained to improve itself, tuning the keypoints into distinctive landmarks. To this end, we propose an iterative algorithm that alternates between producing new pseudo-labels through feature clustering and learning distinctive features for each pseudo-class through contrastive learning.…
Tencent AI Lab（腾讯人工智能实验室）；Tsinghua University（清华大学）
Existing methods for face image manipulation generally focus on editing the expression, changing some predefined attributes, or applying different filters. However, users lack the flexibility of controlling the shapes of different semantic facial parts in the generated face. In this paper, we propose an approach to compute a disentangled shape representation for a face image, namely the FaceShapeGene. The proposed FaceShapeGene encodes the shape information of each semantic facial part separately into a 1D latent vector. On the basis of the FaceShapeGene, a novel part-wise face image editing s…
Vector Quantized-Elites: Unsupervised and Problem-Agnostic Quality-Diversity Optimization
Computational Intelligence Laboratory (CILab), Department of Mathematics, University of Patras（数学系计算智能实验室（CILab），帕特拉大学）；Laboratory of Automation and Robotics (LAR) in the Department of Electrical & Computer Engineering, University of Patras（电气与计算机工程系自动化与机器人实验室（LAR），帕特拉大学）
The variational autoencoder (VAE) can learn the manifold of natural images on certain datasets, as evidenced by meaningful interpolating or extrapolating in the continuous latent space. However, on discrete data such as text, it is unclear if unsupervised learning can discover similar latent space that allows controllable manipulation. In this work, we find that sequence VAEs trained on text fail to properly decode when the latent codes are manipulated, because the modified codes often land in holes or vacant regions in the aggregated posterior latent space, where the decoding network fails to…
UDCR: Unsupervised Aortic DSA/CTA Rigid Registration Using Deep Reinforcement Learning and Overlap Degree Calculation
School of Artificial Intelligence, Beijing University of Posts and Telecommunications（北京邮电大学人工智能学院）；Shanghai Changhai Hospital, Naval Medical University（海军军医大学上海长海医院）；Dalian University of Technology（大连理工大学）；School of Computer Science and Information Security, Guilin University of Electronic Technology（桂林电子科技大学计算机科学与信息安全学院）；The Fourth Hospital of Hebei Medical University and Hebei Tumor Hospital（河北医科大学第四医院暨河北省肿瘤医院）；Erasmus MC, University Medical Center Rotterdam（鹿特丹伊拉斯姆斯大学医学中心）
Korea Advanced Institute of Science and Technology（韩国科学技术院）；The Catholic University of Korea（韩国加图立大学）；University of Ulsan College of Medicine（蔚山大学医学院）；Asan Medical Center（峨山医疗中心）；Korea Automotive Technology Institute（韩国汽车技术研究院）
Image registration is a fundamental task in medical image analysis. Recently, deep learning based image registration methods have been extensively investigated due to their excellent performance despite the ultra-fast computational time. However, the existing deep learning methods still have limitation in the preservation of original topology during the deformation with registration vector fields. To address this issues, here we present a cycle-consistent deformable image registration. The cycle consistency enhances image registration performance by providing an implicit regularization to pres…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Seoul National University College of Medicine, Seoul National University Bundang Hospital（首尔大学医学院，首尔大学盆唐医院）
Time-of-flight magnetic resonance angiography (TOF-MRA) is one of the most widely used non-contrast MR imaging methods to visualize blood vessels, but due to the 3-D volume acquisition highly accelerated acquisition is necessary. Accordingly, high quality reconstruction from undersampled TOF-MRA is an important research topic for deep learning. However, most existing deep learning works require matched reference data for supervised training, which are often difficult to obtain. By extending the recent theoretical understanding of cycleGAN from the optimal transport theory, here we propose a no…
Korea Advanced Institute of Science and Technology (KAIST)（韩国科学技术院）；Seoul National University College of Medicine（首尔大学医学院）
Diffusion-weighted MRI is nowadays performed routinely due to its prognostic ability, yet the quality of the scans are often unsatisfactory which can subsequently hamper the clinical utility. To overcome the limitations, here we propose a fully unsupervised quality enhancement scheme, which boosts the resolution and removes the motion artifact simultaneously. This process is done by first training the network using optimal transport driven cycleGAN with stochastic degradation block which learns to remove aliasing artifacts and enhance the resolution, then using the trained network in the test …
To conduct a radiomics or deep learning research experiment, the radiologists or physicians need to grasp the needed programming skills, which, however, could be frustrating and costly when they have limited coding experience. In this paper, we present DARWIN, a flexible research platform with a graphical user interface for medical imaging research. Our platform is consists of a radiomics module and a deep learning module. The radiomics module can extract more than 1000 dimension features(first-, second-, and higher-order) and provided many draggable supervised and unsupervised machine learnin…
Jiangnan University（江南大学）；Nanjing University of Science and Technology（南京理工大学）
This paper focuses on hyperspectral image (HSI) super-resolution that aims to fuse a low-spatial-resolution HSI and a high-spatial-resolution multispectral image to form a high-spatial-resolution HSI (HR-HSI). Existing deep learning-based approaches are mostly supervised that rely on a large number of labeled training samples, which is unrealistic. The commonly used model-based approaches are unsupervised and flexible but rely on hand-craft priors. Inspired by the specific properties of model, we make the first attempt to design a model inspired deep network for HSI super-resolution in an unsu…
Single-particle cryo-electron microscopy (cryo-EM) is a leading technology to resolve the structure of molecules. Early in the process, the user detects potential particle images in the raw data. Typically, there are many false detections as a result of high levels of noise and contamination. Currently, removing the false detections requires human intervention to sort the hundred thousands of images. We propose a statistically-established unsupervised algorithm to remove non-particle images. We model the particle images as a union of low-dimensional subspaces, assuming non-particle images are …
