US-JEPA: A Joint Embedding Predictive Architecture for Ultrasound
Biomedical AI Research Lab, University of California, Los Angeles, USA(生物医学人工智能研究实验室,加州大学洛杉矶分校);Department of Bioengineering, University of California, Los Angeles, USA(生物工程系,加州大学洛杉矶分校);Medical Informatics Home Area, University of California, Los Angeles, USA(医学信息学家庭区域,加州大学洛杉矶分校);Department of Electrical and Computer Engineering, University of California, Los Angeles, USA(电气与计算机工程系,加州大学洛杉矶分校);Department of Pathology, University of California, Los Angeles, USA(病理学系,加州大学洛杉矶分校);Department of Radiology, University of California, Los Angeles, USA(放射学系,加州大学洛杉矶分校)
Ashwath Radhachandran, Vedrana Ivezić, Shreeram Athreya, Corey W. Arnold, William Speier
Ultrasound (US) imaging poses unique challenges for representation learning due to its inherently noisy acquisition process. The low signal-to-noise ratio and stochastic speckle patterns hinder standard self-supervised learning methods relying on a pixel-level reconstruction objective. Joint-Embedding Predictive Architectures (JEPAs) address this drawback by predicting masked latent representations rather than raw pixels. However, standard approaches depend on hyperparameter-brittle and computationally expensive online teachers updated via exponential moving average. We propose US-JEPA, a self-supervised framework that adopts the Static-teacher Asymmetric Latent Training (SALT) objective. By using a frozen, domain-specific teacher to provide stable latent targets, US-JEPA decouples student-teacher optimization and pushes the student to expand upon the semantic priors of the teacher. In addition, we provide the first rigorous comparison of all publicly available state-of-the-art ultrasound foundation models on UltraBench, a public dataset benchmark spanning multiple organs and pathological conditions. Under linear probing for diverse classification tasks, US-JEPA achieves performance competitive with or superior to domain-specific and universal vision foundation model baselines. Our results demonstrate that masked latent prediction provides a stable and efficient path toward robust ultrasound representations.
MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Features
Meta AI;Inria(法国国家信息与自动化研究所);École normale supérieure(巴黎高等师范学院);CNRS(法国国家科学研究中心);PSL Research University(巴黎文理研究大学);Courant Institute, New York University(纽约大学柯朗研究所);Center for Data Science, New York University(纽约大学数据科学中心)
From Alignment to Prediction: A Study of Self-Supervised Learning and Predictive Representation Learning
Department of Information and Communication Technology, School of Technology, Pandit Deendayal Energy University, Gandhinagar, Gujarat 382007, India(信息与通信技术系,技术学院,潘迪·德恩达尔能源大学,古吉拉特邦加尔各纳,382007,印度)
JetParticle-JEPA: An Efficient Self-Supervised Representation Learning method for Jet Tagging in High-Energy Physics
GREYC, Normandy University, Unicaen, ENSICAEN, UMR CNRS 6072(GREYC,诺曼底大学,Unicaen,ENSICAEN,CNRS UMR 6072);LPC, Normandy University, Unicaen, ENSICAEN, IN2P3, UMR CNRS 6534(LPC,诺曼底大学,Unicaen,ENSICAEN,IN2P3,CNRS UMR 6534)
JEPA as a Neural Tokenizer: Learning Robust Speech Representations with Density Adaptive Attention
Carnegie Mellon University(卡内基梅隆大学);Amazon GenAI(亚马逊生成人工智能);James Silberrad Brown Center for Artificial Intelligence(詹姆斯·西伯拉德·布朗人工智能中心);University of Bristol(布里斯托大学);Stanford University(斯坦福大学);Northeastern University(东北大学);New York University(纽约大学)
CF-JEPA: Mask-free forward prediction with asymmetric encoder utilization for time-series representation learning
Graduate School of Artificial Intelligence Convergence Engineering, Changwon National University(昌原国立大学人工智能融合工程研究生院);Department of Artificial Intelligence Engineering, Changwon National University(昌原国立大学人工智能工程系)
University of British Columbia(不列颠哥伦比亚大学);Vector Institute for AI(人工智能向量研究所);University of Fukui(福井大学);Massachusetts Institute of Technology(麻省理工学院);University of Strasbourg(斯特拉斯堡大学)