一种轻量级的用于基于能量的联合嵌入预测架构库
A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures
AI总结：
本文提出EB-JEPA库，展示如何将图像级自监督学习的表示学习技术扩展到视频和动作条件世界模型，通过实验验证其在任务中的有效性。
摘要：
我们提出了EB-JEPA，一个用于学习表示和世界模型的开源库，使用联合嵌入预测架构（JEPAs）。JEPAs在表示空间而非像素空间中进行预测，避免了生成建模的弊端，同时捕捉到语义有意义的特征，适用于下游任务。我们的库提供了模块化、自包含的实现，展示了图像级自监督学习的表示学习技术如何扩展到视频，其中时间动态增加了复杂性，以及最终到动作条件的世界模型，其中模型必须进一步学习预测控制输入的效果。每个示例都设计为在单个GPU上几小时内训练，使基于能量的自监督学习对研究和教育可及。我们对JEA组件在CIFAR-10上的消融研究显示，91%的准确率表明模型学习了有用的特征。扩展到视频，我们包括一个关于Moving MNIST的多步预测示例，展示了相同原则如何扩展到时间建模。最后，我们展示了这些表示如何驱动动作条件的世界模型，在Two Rooms导航任务中达到97%的规划成功率。全面的消融研究揭示了每个正则化组件对防止表示崩溃的重要性。代码可在https://github.com/facebookresearch/eb_jepa获得。
英文摘要：
We present EB-JEPA, an open-source library for learning representations and world models using Joint-Embedding Predictive Architectures (JEPAs). JEPAs learn to predict in representation space rather than pixel space, avoiding the pitfalls of generative modeling while capturing semantically meaningful features suitable for downstream tasks. Our library provides modular, self-contained implementations that illustrate how representation learning techniques developed for image-level self-supervised learning can transfer to video, where temporal dynamics add complexity, and ultimately to action-conditioned world models, where the model must additionally learn to predict the effects of control inputs. Each example is designed for single-GPU training within a few hours, making energy-based self-supervised learning accessible for research and education. We provide ablations of JEA components on CIFAR-10. Probing these representations yields 91% accuracy, indicating that the model learns useful features. Extending to video, we include a multi-step prediction example on Moving MNIST that demonstrates how the same principles scale to temporal modeling. Finally, we show how these representations can drive action-conditioned world models, achieving a 97% planning success rate on the Two Rooms navigation task. Comprehensive ablations reveal the critical importance of each regularization component for preventing representation collapse. Code is available at https://github.com/facebookresearch/eb_jepa.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
