Vision-Language Synthetic Data Enhances Echocardiography Downstream Tasks
School of Biomedical Engineering, University of British Columbia（不列颠哥伦比亚大学生物医学工程学院）；Department of Electrical and Computer Engineering, University of British Columbia（不列颠哥伦比亚大学电气与计算机工程系）；Department of Orthopaedics, University of British Columbia（不列颠哥伦比亚大学骨科）；Department of Radiology, University of British Columbia（不列颠哥伦比亚大学放射学系）；Department of Medicine, University of British Columbia（不列颠哥伦比亚大学医学系）
Pooria Ashrafian, Milad Yazdani, Moein Heidari, Dena Shahriari, Ilker Hacihaliloglu
High-quality, large-scale data is essential for robust deep learning models in medical applications, particularly ultrasound image analysis. Diffusion models facilitate high-fidelity medical image generation, reducing the costs associated with acquiring and annotating new images. This paper utilizes recent vision-language models to produce diverse and realistic synthetic echocardiography image data, preserving key features of the original images guided by textual and semantic label maps. Specifically, we investigate three potential avenues: unconditional generation, generation guided by text, and a hybrid approach incorporating both textual and semantic supervision. We show that the rich contextual information present in the synthesized data potentially enhances the accuracy and interpretability of downstream tasks, such as echocardiography segmentation and classification with improved metrics and faster convergence. Our implementation with checkpoints, prompts, and the created synthetic dataset will be publicly available at \href{https://github.com/Pooria90/DiffEcho}{GitHub}.
A Data Augmentation Pipeline to Generate Synthetic Labeled Datasets of 3D Echocardiography Images using a GAN
GE Vingmed Ultrasound - GE Healthcare（GE Vingmed超声 - GE医疗）；CARIM School for Cardiovascular Diseases（CARIM心血管疾病学院）；Maastricht University Medical Center（马斯特里赫特大学医学中心）
Echocardiography to Cardiac MRI View Transformation for Real-Time Blind Restoration
Tampere University（坦佩雷大学）；Qatar University（卡塔尔大学）；VTT Technical Research Centre of Finland（芬兰国家技术研究中心）；Hamad Medical Corporation Hospital（哈马德医疗公司医院）；Hayatabad Medical Complex（海亚特阿巴德医疗中心）
High-Fidelity Synthetic Transmission Electron Microscopy Image Generation Using Diffusion Probabilistic Models for Data-Limited Semiconductor Metrology
Chemnitz University of Technology, Chemnitz, Germany（化学工业大学，化学矿泉，德国）；Interuniversity Microelectronics Centre (imec), Leuven, Belgium（大学微电子中心（imec），卢汶，比利时）
SkinGenBench: Generative Model and Preprocessing Effects for Synthetic Dermoscopic Augmentation in Melanoma Diagnosis
Alliance School of Advanced Computing, Alliance University, India（印度联盟大学高级计算学院）；Department of Computer Science and Communication, Østfold University College, 1783 Halden, Norway（挪威东福尔大学学院计算机科学与传播系）；Department of Dentistry and Oral Health, Aarhus University, 8000 Aarhus, Denmark（丹麦奥胡斯大学牙科与口腔健康系）
Towards Application Aligned Synthetic Surgical Image Synthesis
Department of Translational Surgical Oncology, NCT/UCC Dresden（转化外科肿瘤学系）；DKFZ, Faculty of Medicine and University Hospital Carl Gustav Carus, TUD Dresden,HZDR, Germany（德累斯顿技术大学医学院）；The Centre for Tactile Internet with Human-in-the-Loop (CeTI), TUD Dresden（人机交互 tactile 网络中心）
Institute for Applied Computational Science, Harvard University（哈佛大学应用计算科学研究所）；Microsoft AI for Good Research Lab（微软人工智能向善研究实验室）
In this paper we propose a mask-conditional synthetic image generation model for creating synthetic satellite imagery datasets. Given a dataset of real high-resolution images and accompanying land cover masks, we show that it is possible to train an upstream conditional synthetic imagery generator, use that generator to create synthetic imagery with the land cover masks, then train a downstream model on the synthetic imagery and land cover masks that achieves similar test performance to a model that was trained with the real imagery. Further, we find that incorporating a mixture of real and sy…
Problem: There is a lack of big data for the training of deep learning models in medicine, characterized by the time cost of data collection and privacy concerns. Generative adversarial networks (GANs) offer both the potential to generate new data, as well as to use this newly generated data, without inclusion of patients' real data, for downstream applications. Approach: A series of GANs were trained and applied for a downstream computer vision spine radiograph abnormality classification task. Separate classifiers were trained with either access or no access to the original imaging. Trained G…
Improving the Effectiveness of Deep Generative Data
Bosch Center for Artificial Intelligence（博世人工智能中心）；Institute of Industrial Manufacturing and Management IFF, University of Stuttgart（斯图加特大学工业制造与管理研究所）；Fraunhofer Institute for Manufacturing Engineering and Automation (IPA)（弗劳恩霍夫制造工程与自动化研究所）
The ability to generate privacy-preserving synthetic versions of sensitive image datasets could unlock numerous ML applications currently constrained by data availability. Due to their astonishing image generation quality, diffusion models are a prime candidate for generating high-quality synthetic data. However, recent studies have found that, by default, the outputs of some diffusion models do not preserve training data privacy. By privately fine-tuning ImageNet pre-trained diffusion models with more than 80M parameters, we obtain SOTA results on CIFAR-10 and Camelyon17 in terms of both FID …
Rethinking Driving World Model as Synthetic Data Generator for Perception Tasks
Peking University（北京大学）；Xiaomi EV（小米电动车）；Huazhong University of Science and Technology（华中科技大学）；Beijing Key Laboratory of Data Intelligence and Security (Peking University)（北京数据智能与安全重点实验室（北京大学））；Zhongguancun Academy（中关村学院）
NASA Ames Research Center（美国国家航空航天局艾姆斯研究中心）；Bay Area Environmental Research Institute（湾区环境研究所）；Microsoft Research AI（微软研究院人工智能）；California State University Monterey Bay（加利福尼亚州立大学蒙特雷湾分校）
Earth observing satellites carrying multi-spectral sensors are widely used to monitor the physical and biological states of the atmosphere, land, and oceans. These satellites have different vantage points above the earth and different spectral imaging bands resulting in inconsistent imagery from one to another. This presents challenges in building downstream applications. What if we could generate synthetic bands for existing satellites from the union of all domains? We tackle the problem of generating synthetic spectral imagery for multispectral sensors as an unsupervised image-to-image trans…
NAAMII（尼泊尔人工智能与信息研究所）；Imperial College London（伦敦帝国理工学院）；KAIST（韩国科学技术院）
Multi-domain image-to-image translation with conditional Generative Adversarial Networks (GANs) can generate highly photo realistic images with desired target classes, yet these synthetic images have not always been helpful to improve downstream supervised tasks such as image classification. Improving downstream tasks with synthetic examples requires generating images with high fidelity to the unknown conditional distribution of the target class, which many labeled conditional GANs attempt to achieve by adding soft-max cross-entropy loss based auxiliary classifier in the discriminator. As rece…
Scientific Image Synthesis: Benchmarking, Methodologies, and Downstream Utility
Shanghai Jiao Tong University（上海交通大学）；OpenDataLab, Shanghai Artificial Intelligence Laboratory（OpenDataLab，上海人工智能实验室）；The University of Hong Kong（香港大学）；Peking University（北京大学）
Generative Adversarial Networks (GANs) have made releasing of synthetic images a viable approach to share data without releasing the original dataset. It has been shown that such synthetic data can be used for a variety of downstream tasks such as training classifiers that would otherwise require the original dataset to be shared. However, recent work has shown that the GAN models and their synthetically generated data can be used to infer the training set membership by an adversary who has access to the entire dataset and some auxiliary information. Current approaches to mitigate this problem…
Phase Unwrapping of Color Doppler Echocardiography using Deep Learning
CREATIS（CREATIS实验室）；CNRS（法国国家科学研究中心）；INSERM（法国国家健康与医学研究院）；INSA Lyon（里昂国立应用科学学院）；University of Lyon 1（里昂第一大学）；Institut Universitaire de France（法兰西大学研究院）
Improving Performance, Robustness, and Fairness of Radiographic AI Models with Finely-Controllable Synthetic Data
Center for Artificial Intelligence in Medicine and Imaging, Stanford University（斯坦福大学人工智能医学与成像中心）；Department of Radiology, Stanford University（斯坦福大学放射科）；Department of Applied Physics, Stanford University（斯坦福大学应用物理系）；Department of Biomedical Data Science, Stanford University（斯坦福大学生物医学数据科学系）；Institute for Diagnostic and Interventional Radiology, University Hospital Zurich, University of Zurich（苏黎世大学医院诊断与介入放射学研究所）；LAION；Juelich Supercomputing Center (JSC), Research Center Juelich (FZJ)（耶鲁超级计算中心（JSC）、耶鲁研究中心（FZJ））；Department of Radiology & Imaging Sciences, Emory University（埃默里大学放射科与成像科学系）；Department of Radiology, University of Florida College of Medicine（佛罗里达大学医学院放射科）
Breast Ultrasound Tumor Generation via Mask Generator and Text-Guided Network:A Clinically Controllable Framework with Downstream Evaluation
School of Biomedical Engineering, Shenzhen University Medical School, Shenzhen University, Shenzhen, China（生物医学工程学院，深圳大学医学院，深圳大学，深圳，中国）；Smart Medical Imaging, Learning and Engineering (SMILE) Lab, Shenzhen University, Shenzhen, China（智能医学影像、学习与工程（SMILE）实验室，深圳大学，深圳，中国）；Nuclear Medicine Department, the Seventh Affiliated Hospital, Sun Yat-Sen University, Shenzhen, China（核医学科，中山大学第七附属医院，中山大学，深圳，中国）；Department of Radiology, the Seventh Affiliated Hospital, Sun Yat-Sen University, Shenzhen, China（放射科，中山大学第七附属医院，中山大学，深圳，中国）；Shenzhen University of Advanced Technology, Shenzhen, China（深圳大学先进技术大学，深圳，中国）
A Task-Driven Evaluation of UAV Detection and Tracking under Synthetic Fog
Center for Aerospace Research (CfAR), University of Victoria（维多利亚大学航空航天研究中心）；Advanced Control and Intelligent Systems (ACIS) Lab, University of Victoria（维多利亚大学先进控制与智能系统实验室）
Seg-CycleGAN : SAR-to-optical image translation guided by a downstream task
Gina Cody School of Engineering and Computer Science, Concordia University（康考迪亚大学吉娜·科迪工程与计算机科学学院）；School of Automation, Northwestern Polytechnical University（西北工业大学自动化学院）；School of Cybersecurity, Northwestern Polytechnical University（西北工业大学网络空间安全学院）
