发表机构
University of Toronto; University of Waterloo; University of Alberta; University of Sussex（多伦多大学; 滑铁卢大学; 阿尔伯塔大学; 萨塞克斯大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究提出适配3D脑部MRI的Med-D-JEPA模型，在生成质量、分类与分割任务上均优于或相当基线，证明联合嵌入预测生成可用于3D医学图像合成。
AI 中文摘要
联合嵌入预测架构（JEPAs）主要是为自监督表示学习而开发的。去噪JEPAs（D-JEPA）最近在自然图像上展现出强大的生成能力，但其在3D医学成像中的适用性仍未被探索。基于D-JEPA框架，我们提出了Med-D-JEPA，这是针对3D脑部MRI的联合嵌入预测生成的系统适配与评估。Med-D-JEPA基于由3D KL正则化对抗变分自编码器生成的连续隐式标记运行，结合了掩码上下文预测、表示级对齐、逐标记扩散以及迭代下一组标记采样。我们在BraTS2019和OASIS-1数据集上评估了无条件和类条件生成质量、下游分类效用，以及在BraTS2020上的初步全肿瘤分割。在不同生成设置下，Med-D-JEPA在保真度和多样性指标上相比多个强基线取得了更优或相当的性能。与使用真实样本训练相比，基于Med-D-JEPA的合成预训练将BraTS2019上的分类AUC从0.63提升至0.85，OASIS-1上的分类AUC从0.78提升至0.87。在分割研究中，使用Med-D-JEPA样本进行预训练将Dice分数从0.74提升至0.80，HD95从13.40 mm降低至9.56 mm。这些发现确立了联合嵌入预测生成是3D医学图像合成的一个有前景方向，并鼓励在该方向开展进一步研究。
英文摘要
Joint-embedding predictive architectures (JEPAs) have primarily been developed for self-supervised representation learning. Denoising JEPA (D-JEPA) recently demonstrated strong generative capabilities on natural images, yet the applicability to 3D medical imaging remains unexplored. Building on the D-JEPA framework, we present Med-D-JEPA, a systematic adaptation and evaluation of joint-embedding predictive generation for 3D brain MRI. Med-D-JEPA operates on continuous latent tokens produced by a 3D KL-regularized adversarial variational autoencoder, and combines masked context prediction, representation-level alignment, per-token diffusion, and iterative next-set-of-token sampling. We evaluate unconditional and class-conditional generation quality on BraTS2019 and OASIS-1 datasets; downstream classification utility; and preliminary whole-tumor segmentation on BraTS2020. Across different generation settings, Med-D-JEPA achieves superior or competitive performance compared to several strong baselines on fidelity and diversity metrics. Compared to training with real samples, Med-D-JEPA-based synthetic pretraining improves classification AUC from 0.63 to 0.85 on BraTS2019 and from 0.78 to 0.87 on OASIS-1. In the segmentation study, pretraining on Med-D-JEPA samples improves Dice from 0.74 to 0.80 and reduces HD95 from 13.40 to 9.56 mm. These findings establish joint-embedding predictive generation as a promising direction for 3D medical image synthesis and encourage further research in this direction.
CommentsPreprint, code will be released after review
