首页/论文/2010.03468

arXiv 2010.034682020-10-07eess.IVcs.CVcs.LG

Discriminative Cross-Modal Data Augmentation for Medical Imaging Applications

University of California San Diego（加利福尼亚大学圣地亚哥分校）

Yue Yang, Pengtao Xie

英文摘要：

While deep learning methods have shown great success in medical image analysis, they require a number of medical images to train. Due to data privacy concerns and unavailability of medical annotators, it is oftentimes very difficult to obtain a lot of labeled medical images for model training. In this paper, we study cross-modality data augmentation to mitigate the data deficiency issue in the medical imaging domain. We propose a discriminative unpaired image-to-image translation model which translates images in source modality into images in target modality where the translation task is conducted jointly with the downstream prediction task and the translation is guided by the prediction. Experiments on two applications demonstrate the effectiveness of our method.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

阅读原文

阅读 PDF ↗

arXiv 页面 ↗HTML 原文 ↗

本文内容

英文摘要相关论文

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

继续探索

相关论文

CLIP中的语言增强用于改进多模态医学图像上的解剖结构检测

Language Augmentation in CLIP for Improved Anatomy Detection on Multi-modal Medical Images

Indian Institute of Technology Madras（马德拉斯印度理工学院）；GE HealthCare（GE医疗）

本文利用CLIP模型，通过基线微调、添加身体部位超集及图像与语言增强等方法，在多模态MR和CT图像中自动生成全身标准化器官列表，相比PubMedCLIP性能提升47.6%。

2024-05-31cs.CV

X-TRA: Improving Chest X-ray Tasks with Cross-Modal Retrieval Augmentation

University of Amsterdam（阿姆斯特丹大学）

An important component of human analysis of medical images and their context is the ability to relate newly seen things to related instances in our memory. In this paper we mimic this ability by using multi-modal retrieval augmentation and apply it to several tasks in chest X-ray analysis. By retrieving similar images and/or radiology reports we expand and regularize the case at hand with additional knowledge, while maintaining factual knowledge consistency. The method consists of two components. First, vision and language modalities are aligned using a pre-trained CLIP model. To enforce that …

2023-02-22cs.CV

OCTAve: 2D en face Optical Coherence Tomography Angiography Vessel Segmentation in Weakly-Supervised Learning with Locality Augmentation

Vidyasirimedhi Institute of Science & Technology（维迪亚西里梅迪科技学院）；Kamnoetvidya Science Academy（卡姆诺维迪亚科学学院）；Ningbo Institute of Industrial Technology, Chinese Academy of Sciences（中国科学院宁波工业技术研究院）；Cixi Institute of Biomedical Engineering（慈溪生物医学工程研究所）；Kasetsart University（泰国农业大学）；Nanyang Technological University（南洋理工大学）

While there have been increased researches using deep learning techniques for the extraction of vascular structure from the 2D en face OCTA, for such approach, it is known that the data annotation process on the curvilinear structure like the retinal vasculature is very costly and time consuming, albeit few tried to address the annotation problem. In this work, we propose the application of the scribble-base weakly-supervised learning method to automate the pixel-level annotation. The proposed method, called OCTAve, combines the weakly-supervised learning using scribble-annotated ground truth …

2022-07-25cs.CVcs.LGeess.IV

重新审视超声图像的数据增强

Revisiting Data Augmentation for Ultrasound Images

Institut Intelligence et Données (IID)（智能与数据研究所）；Université Laval（拉瓦尔大学）；Mila

本文通过构建覆盖14项超声图像分类与分割任务的标准化基准，系统评估了多种数据增强技术的有效性，发现自然图像常用的增强方法（如TrivialAugment）在超声图像上同样甚至更有效，并提出了可推广到其他模态的结构化评估方法。

2025-07-18cs.CVeess.IV

通过增强驱动泛化实现跨模态医学图像分割的联邦学习

Federated Learning for Cross-Modality Medical Image Segmentation via Augmentation-Driven Generalization

Department of Circulation and Medical Imaging, Norwegian University of Science and Technology（循环医学成像系，挪威科学与技术大学）

本文提出一种联邦学习方法，通过增强驱动泛化实现跨模态医学图像分割，提升模型泛化能力的同时保护数据隐私。

2026-09-10cs.CV

面向数据稀缺场景下精准肿瘤分割的判别式哈密顿变分自编码器

Discriminative Hamiltonian Variational Autoencoder for Accurate Tumor Segmentation in Data-Scarce Regimes

LITIS UR 4108, University of Rouen-Normandy（鲁昂-诺曼底大学 LITIS UR 4108 实验室）；CLCC Henri Becquerel（亨利·贝克勒尔癌症中心）

针对医学图像分割中标注数据稀缺的问题，提出结合HVAE与判别式正则化的端到端混合架构，逐切片生成带掩码的高质量增强样本，在BRATS、HECKTOR数据集上验证了有限数据下的分割有效性。

2024-06-17cs.CVeess.IV

更多相关论文收起更多论文

CAMANet：用于放射学报告生成的类激活图引导注意力网络

CAMANet: Class Activation Map Guided Attention Network for Radiology Report Generation

University of Warwick（华威大学）；NVIDIA AI Tech Center（英伟达人工智能技术中心）；Alan Turing Institute（阿兰·图灵研究所）

提出 CAMANet，利用聚合类激活图监督跨模态注意力并增强判别信息，在两个 RRG 基准上超越已有 SOTA 方法。

2024-03-03cs.CV

FedGIN: 一种用于多模态图像器官分割的联邦学习动态全局强度非线性增强

FedGIN: Federated Learning with Dynamic Global Intensity Non-linear Augmentation for Organ Segmentation using Multi-modal Images

Department of Circulation and Medical Imaging（循证医学与影像学系）；Norwegian University of Science and Technology（挪威科学技术大学）

FedGIN通过动态全局强度非线性增强模块，在不共享原始数据的情况下实现多模态器官分割，提升跨模态泛化能力。

2025-08-07cs.AIcs.CV

面向可变形多模态医学图像配准的模态无关结构图像表示学习

Modality-Agnostic Structural Image Representation Learning for Deformable Multi-Modality Medical Image Registration

DAMO Academy, Alibaba Group（阿里巴巴达摩院）；Hupan Lab（湖畔实验室）；College of Computer Science and Technology, Zhejiang University, China（浙江大学计算机科学与技术学院）；Shengjing Hospital of China Medical University, China（中国医科大学附属盛京医院）

本文提出模态无关的结构表示学习方法，结合深度邻域自相似性和解剖感知对比学习，在无需标注或预对齐的情况下学习判别性深度结构表示，并在多期相CT、腹部MR-CT和脑MR配准中取得优于传统方法的精度。

2024-04-01cs.CV

Discriminative Multi-modality Speech Recognition

Xpeng motors（小鹏汽车）

Vision is often used as a complementary modality for audio speech recognition (ASR), especially in the noisy environment where performance of solo audio modality significantly deteriorates. After combining visual modality, ASR is upgraded to the multi-modality speech recognition (MSR). In this paper, we propose a two-stage speech recognition model. In the first stage, the target voice is separated from background noises with help from the corresponding visual information of lip movements, making the model 'listen' clearly. At the second stage, the audio modality combines visual modality again …

2020-05-13cs.CLcs.CVcs.SD

使用生成式混合增强与自训练的跨模态肿瘤分割

Cross-modal tumor segmentation using generative blending augmentation and self training

LaTIM；Inserm（法国国家健康与医学研究院）；Université de Bretagne Occidentale（西布列塔尼大学）；IMT Atlantique（大西洋电信学院）；CHRU Brest University Hospital（布雷斯特大学中心医院）

本文针对跨模态肿瘤分割中数据稀缺和域偏移问题，提出生成式混合增强（GBA）结合迭代自训练的方法，并在MICCAI CrossMoDA 2022挑战赛前庭神经鞘瘤分割任务中取得第一名。

2024-03-29cs.CVeess.IV

An Unpaired Cross-modality Segmentation Framework Using Data Augmentation and Hybrid Convolutional Networks for Segmenting Vestibular Schwannoma and Cochlea

Huazhong University of Science and Technology（华中科技大学）；Kennesaw State University（肯尼索州立大学）

The crossMoDA challenge aims to automatically segment the vestibular schwannoma (VS) tumor and cochlea regions of unlabeled high-resolution T2 scans by leveraging labeled contrast-enhanced T1 scans. The 2022 edition extends the segmentation task by including multi-institutional scans. In this work, we proposed an unpaired cross-modality segmentation framework using data augmentation and hybrid convolutional networks. Considering heterogeneous distributions and various image sizes for multi-institutional scans, we apply the min-max normalization for scaling the intensities of all scans between …

2022-11-28cs.CVeess.IV

用于细胞核分割的多模态显微镜图像风格迁移

Multi-Modality Microscopy Image Style Transfer for Nuclei Segmentation

Technical University Munich（慕尼黑工业大学）；Helmholtz Zentrum München（慕尼黑亥姆霍兹中心）

本文提出基于GAN且解耦内容与风格的多模态显微镜图像增强方法，可跨细胞检测、光照和成像模态保留结构并转换风格，显著提升Mask R-CNN细胞核分割性能。

2021-11-23cs.CVeess.IVq-bio.QM

Shape-consistent Generative Adversarial Networks for multi-modal Medical segmentation maps

Tel Aviv University（特拉维夫大学）

Image translation across domains for unpaired datasets has gained interest and great improvement lately. In medical imaging, there are multiple imaging modalities, with very different characteristics. Our goal is to use cross-modality adaptation between CT and MRI whole cardiac scans for semantic segmentation. We present a segmentation network using synthesised cardiac volumes for extremely limited datasets. Our solution is based on a 3D cross-modality generative adversarial network to share information between modalities and generate synthesized data using unpaired datasets. Our network utili…

2022-02-04cs.CVcs.LGeess.IV

Unsupervised Cross-Modality Domain Adaptation for Segmenting Vestibular Schwannoma and Cochlea with Data Augmentation and Model Ensemble

Vanderbilt University（范德堡大学）

Magnetic resonance images (MRIs) are widely used to quantify vestibular schwannoma and the cochlea. Recently, deep learning methods have shown state-of-the-art performance for segmenting these structures. However, training segmentation models may require manual labels in target domain, which is expensive and time-consuming. To overcome this problem, domain adaptation is an effective way to leverage information from source domain to obtain accurate segmentations without requiring manual labels in target domain. In this paper, we propose an unsupervised learning framework to segment the VS and c…

2022-08-24cs.CVeess.IV

MODAL：基于模型驱动稀疏解耦与文本-图像差分滤波的多模态对象重识别

MODAL: Multi-Modal Object Re-ID via Model-Driven Sparse Decoupling and Text-Image Differential Filtering

College of Computer Science and Technology, National University of Defense Technology（国防科技大学计算机科学与技术学院）；School of Computer Science and Information Engineering, Hefei University of Technology（合肥工业大学计算机与信息工程学院）

本文提出MODAL多模态对象重识别框架，通过多模态特征稀疏解耦模块与文本-图像差分滤波模块解决现有方法的特征纠缠与模态缺失性能下降问题，在四个数据集上取得SOTA性能。

2026-08-18cs.CVeess.IV