I-ODA, Real-World Multi-modal Longitudinal Data for OphthalmicApplications
University of Illinois at Chicago（伊利诺伊大学芝加哥分校）
Nooshin Mojab, Vahid Noroozi, Abdullah Aleem, Manoj P. Nallabothula, Joseph Baker, Dimitri T. Azar, Mark Rosenblatt, RV Paul Chan, Darvin Yi, Philip S. Yu, Joelle A. Hallak
Data from clinical real-world settings is characterized by variability in quality, machine-type, setting, and source. One of the primary goals of medical computer vision is to develop and validate artificial intelligence (AI) based algorithms on real-world data enabling clinical translations. However, despite the exponential growth in AI based applications in healthcare, specifically in ophthalmology, translations to clinical settings remain challenging. Limited access to adequate and diverse real-world data inhibits the development and validation of translatable algorithms. In this paper, we present a new multi-modal longitudinal ophthalmic imaging dataset, the Illinois Ophthalmic Database Atlas (I-ODA), with the goal of advancing state-of-the-art computer vision applications in ophthalmology, and improving upon the translatable capacity of AI based applications across different clinical settings. We present the infrastructure employed to collect, annotate, and anonymize images from multiple sources, demonstrating the complexity of real-world retrospective data and its limitations. I-ODA includes 12 imaging modalities with a total of 3,668,649 ophthalmic images of 33,876 individuals from the Department of Ophthalmology and Visual Sciences at the Illinois Eye and Ear Infirmary of the University of Illinois Chicago (UIC) over the course of 12 years.
Starting from childhood, the human brain restructures and rewires throughout life. Characterizing such complex brain development requires effective analysis of longitudinal and multi-modal neuroimaging data. Here, we propose such an analysis approach named Longitudinal Correlation Analysis (LCA). LCA couples the data of two modalities by first reducing the input from each modality to a latent representation based on autoencoders. A self-supervised strategy then relates the two latent spaces by jointly disentangling two directions, one in each space, such that the longitudinal changes in latent…
Istituto Italiano di Tecnologia（意大利理工学院）；University of Genoa（热那亚大学）；Fondation Campus Biotech Geneva（日内瓦校园生物技术基金会）；San Raffaele Scientific Institute（圣拉斐尔科学研究所）；University of Verona（维罗纳大学）；Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
In this paper, we present an automated approach for segmenting multiple sclerosis (MS) lesions from multi-modal brain magnetic resonance images. Our method is based on a deep end-to-end 2D convolutional neural network (CNN) for slice-based segmentation of 3D volumetric data. The proposed CNN includes a multi-branch downsampling path, which enables the network to encode information from multiple modalities separately. Multi-scale feature fusion blocks are proposed to combine feature maps from different modalities at different stages of the network. Then, multi-scale feature upsampling blocks ar…
Multi-View Imputation and Cross-Attention Network Based on Incomplete Longitudinal and Multimodal Data for Conversion Prediction of Mild Cognitive Impairment
Longitudinal Multimodal Transformer Integrating Imaging and Latent Clinical Signatures From Routine EHRs for Pulmonary Nodule Classification
Vanderbilt University（范德堡大学）；Siemens Healthineers（西门子医疗）；Saint Luke’s Mid America Heart Institute（圣卢克中美洲心脏研究所）；Vanderbilt University Medical Center（范德堡大学医学中心）
Deep Learning for Longitudinal Medical Imaging: A Scoping Review
Mass General Brigham（麻省总医院布莱根医疗体系）；Harvard Medical School（哈佛医学院）；Dana-Farber Cancer Institute（丹娜-法伯癌症研究所）；Brigham and Women’s Hospital（布莱根妇女医院）；University of California, San Francisco（加州大学旧金山分校）；Inselspital, Bern University Hospital（伯尔尼大学医院（ Inselspital ））；University of Bern（伯尔尼大学）
To study neurodegenerative diseases, longitudinal studies are carried on volunteer patients. During a time span of several months to several years, they go through regular medical visits to acquire data from different modalities, such as biological samples, cognitive tests, structural and functional imaging. These variables are heterogeneous but they all depend on the patient's health condition, meaning that there are possibly unknown relationships between all modalities. Some information may be specific to some modalities, others may be complementary, and others may be redundant. Some data ma…
Technical University of Munich（慕尼黑工业大学）；Shanghai Jiao Tong University（上海交通大学）；Helmholtz Zentrum München（慕尼黑亥姆霍兹中心）；rechts der Isar University Hospital（伊萨尔河右岸大学医院）
Ultrasound (US) is one of the most common medical imaging modalities since it is radiation-free, low-cost, and real-time. In freehand US examinations, sonographers often navigate a US probe to visualize standard examination planes with rich diagnostic information. However, reproducibility and stability of the resulting images often suffer from intra- and inter-operator variation. Reinforcement learning (RL), as an interaction-based learning method, has demonstrated its effectiveness in visual navigating tasks; however, RL is limited in terms of generalization. To address this challenge, we pro…
A Surface-based Multimodal Framework for Multitask Analysis in Alzheimer's Disease
Stevens Neuroimaging and Informatics Institute, Keck School of Medicine, University of Southern California (USC)（南加州大学凯克医学院史蒂恩斯神经影像与信息学研究所）；Alfred E. Mann Department of Biomedical Engineering, Viterbi School of Engineering, University of Southern California (USC)（南加州大学维特比工程学院阿尔弗雷德·E·曼生物医学工程系）；Ming Hsieh Department of Electrical and Computer Engineering, Viterbi School of Engineering, University of Southern California (USC)（南加州大学维特比工程学院明海电气与计算机工程系）
提出一种增强的球形数据驱动多模态框架，通过球形扩散模型生成配对皮层厚度和Tau PET SUVR数据，结合对比学习和上下文学习，实现多任务分类与回归，在ADNI数据集上优于六种基线模型。
