Using generative AI to investigate medical imagery models and datasets
Google（谷歌）；Advanced Clinical（先进临床）；Pro Unlimited；University of California San Francisco（加州大学旧金山分校）；Weizmann Institute of Science（魏茨曼科学研究所）
Oran Lang, Doron Yaya-Stupp, Ilana Traynis, Heather Cole-Lewis, Chloe R. Bennett, Courtney Lyles, Charles Lau, Michal Irani, Christopher Semturs, Dale R. Webste… 展开作者
Oran Lang, Doron Yaya-Stupp, Ilana Traynis, Heather Cole-Lewis, Chloe R. Bennett, Courtney Lyles, Charles Lau, Michal Irani, Christopher Semturs, Dale R. Webster, Greg S. Corrado, Avinatan Hassidim, Yossi Matias, Yun Liu, Naama Hammel, Boris Babenko
AI models have shown promise in many medical imaging tasks. However, our ability to explain what signals these models have learned is severely lacking. Explanations are needed in order to increase the trust in AI-based models, and could enable novel scientific discovery by uncovering signals in the data that are not yet known to experts. In this paper, we present a method for automatic visual explanations leveraging team-based expertise by generating hypotheses of what visual signals in the images are correlated with the task. We propose the following 4 steps: (i) Train a classifier to perform a given task (ii) Train a classifier guided StyleGAN-based image generator (StylEx) (iii) Automatically detect and visualize the top visual attributes that the classifier is sensitive towards (iv) Formulate hypotheses for the underlying mechanisms, to stimulate future research. Specifically, we present the discovered attributes to an interdisciplinary panel of experts so that hypotheses can account for social and structural determinants of health. We demonstrate results on eight prediction tasks across three medical imaging modalities: retinal fundus photographs, external eye photographs, and chest radiographs. We showcase examples of attributes that capture clinically known features, confounders that arise from factors beyond physiological mechanisms, and reveal a number of physiologically plausible novel attributes. Our approach has the potential to enable researchers to better understand, improve their assessment, and extract new knowledge from AI-based models. Importantly, we highlight that attributes generated by our framework can capture phenomena beyond physiology or pathophysiology, reflecting the real world nature of healthcare delivery and socio-cultural factors. Finally, we intend to release code to enable researchers to train their own StylEx models and analyze their predictive tasks.
Benchmarking Differentially Private Residual Networks for Medical Imagery
OpenMined；Manifold Computing Group（曼尼福德计算集团）；Ford R&A（福特研发部门）；Harvard University（哈佛大学）；Indian Institute of Technology Jodhpur（印度理工学院焦特布尔分校）；University of Oxford（牛津大学）
Cyclic Generative Adversarial Networks With Congruent Image-Report Generation For Explainable Medical Image Analysis
Indian Institute of Technology (IIT) Ropar（印度理工学院罗巴尔分校）；Monash University（莫纳什大学）；University of Canberra（堪培拉大学）；Inception Institute of Artificial Intelligence（因塞普申人工智能研究院）
Physics-Inspired Generative Models in Medical Imaging: A Review
KTH Royal Institute of Technology（瑞典皇家理工学院）；Karolinska University Hospital（卡罗林斯卡大学医院）；University of Regensburg（雷根斯堡大学）；Fraunhofer Institute for Digital Medicine MEVIS（弗劳恩霍夫数字医学MEVIS研究所）；Rensselaer Polytechnic Institute（伦斯勒理工学院）
NSD-Imagery: A benchmark dataset for extending fMRI vision decoding methods to mental imagery
University of Minnesota（明尼苏达大学）；Alljoined；Princeton Neuroscience Institute（普林斯顿神经科学研究所）；Stability AI/Medical AI Research Center (MedARC)（Stability AI/医疗AI研究中心（MedARC））
Towards Data-Efficient Medical Imaging: A Generative and Semi-Supervised Framework
Dept. of Electrical and Electronic Engineering Imperial College London（电气与电子工程系帝国理工学院伦敦）；Centre for Vision, Speech and Signal Processing University of Surrey（视觉、语音和信号处理中心萨里大学）
GLOWin: A Flow-based Invertible Generative Framework for Learning Disentangled Feature Representations in Medical Images
Technical University of Munich（慕尼黑工业大学）；deepc GmbH（deepc有限公司）；Ludwig Maximilians University Munich（慕尼黑大学）；Kyung Hee University（庆熙大学）；Johns Hopkins University（约翰斯·霍普金斯大学）
Generative Artificial Intelligence in Medical Imaging: Foundations, Progress, and Clinical Translation
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Chinese Academy of Sciences（中国科学院大学）；Macao Polytechnic University（澳门理工大学）
Generative Text-Guided 3D Vision-Language Pretraining for Unified Medical Image Segmentation
University of Science and Technology of China（中国科学技术大学）；Institute of Artificial Intelligence, Hefei Comprehensive National Science Center（合肥综合性国家科学中心人工智能研究院）；Imperial College London（帝国理工学院）
Reducing bias and increasing utility by federated generative modeling of medical images using a centralized adversary
University of British Columbia（不列颠哥伦比亚大学）；Microsoft（微软）
我们介绍了FELICIA（FEDERATED LEARNING WITH A CentralIzed Advertisement），一种支持协作学习的生成机制。特别是，我们展示了数据有限和有偏见的数据所有者如何从其他数据所有者那里受益，同时保持所有来源的数据的私密性。这是医学图像分析中的常见情况，其中隐私立法阻止数据在本地场所之外共享。FELICIA适用于生成对抗网络（GAN）架构的大家族，包括香草和条件GANs，如本工作所示。我们表明，通过使用FELICIA机制，有限的图像样本的数据所有者可以生成高质量的合成图像，具有高实用性，而数据所有者都没有提供访问其数据。共享仅通过一个中央数据库进行，该数据库只能访问合成数据。在这里，效用被定义为在真实测试集上的分类性能。我们使用基准图像数据集（MNIST，CIFAR-10）以及皮肤病变分类任务的医学图像上的几个现实的医疗保健系统上证明了这些好处。通过多个实验，我们表明，即使在最坏的情况下，将FELICIA与真实数据相结合，也能优雅地实现与真实数据相当的性能，而大多数结果显着提高了效用。
GenMix: Combining Generative and Mixture Data Augmentation for Medical Image Classification
School of Electrical Engineering（电气工程学院）；Korea Advanced Institute of Science and Technology（韩国科学技术院）；Department of Software Convergence（软件融合系）；Seoul Women’s University（首尔女子大学）
The Era of Foundation Models in Medical Imaging is Approaching : A Scoping Review of the Clinical Value of Large-Scale Generative AI Applications in Radiology
3D-StyleGAN: A Style-Based Generative Adversarial Network for Generative Modeling of Three-Dimensional Medical Images
Massachusetts General Hospital（麻省总医院）；Harvard Medical School（哈佛医学院）；Massachusetts Institute of Technology（麻省理工学院）；Univ. Lille（里尔大学）；Inserm（法国国家健康与医学研究院）；CHU Lille（里尔大学医院）
