Learning an energy-based model from data samples is a central problem in machine learning. Many recent and popular methods, such as denoising score matching for training energy-based diffusion models, use stochastic interpolants to corrupt data samples at different noise levels indexed by a time variable. This defines a joint density over both the data space and time, and most methods learn its energy through either spatial or temporal differences. We identify distinct failure modes for both of these approaches. To solve them, we propose Spatiotemporal Noise-Contrastive Estimation (stNCE), a framework for learning the energy through joint spatiotemporal differences. stNCE unifies many existing methods and leads to new training objectives. Experiments on images and molecules demonstrate performance competitive with state-of-the-art density estimation methods.
Capital Fund Management(资本基金管理公司);Society of Fellows, Harvard University(哈佛大学 fellows 会);Machine Learning Lab(机器学习实验室);Courant Institute of Mathematical Sciences, New York University(纽约大学应用数学科学研究所)
Stochastic Interpolants: A Unifying Framework for Flows and Diffusions
Center for Cosmology and Particle Physics New York University(纽约大学宇宙与粒子物理中心);Courant Institute of Mathematical Sciences New York University(纽约大学数学科学学院)
SplineFlow: Flow Matching for Dynamical Systems with B-Spline Interpolants
CISPA Helmholtz Center for Information Security(CISPA海德堡信息安全研究中心);Department of Computer Science and Technology(计算机科学与技术系);University of Cambridge(剑桥大学)
Open Materials Generation with Stochastic Interpolants
Center for Soft Matter Research, Department of Physics, New York University, New York 10003, USA;Simons Center for Computational Physical Chemistry, Department of Chemistry, New York University, New York 10003, USA;Department of Aerospace Engineering;Mechanics, University of Minnesota, Minneapolis, MN 55455, USA;Department of Computer Science \& Engineering, University of Minnesota, Minneapolis, MN 55455, USA;Department of Physics, University of Florida, Gainesville, FL 32611, USA;Quantum Theory Project, University of Florida, Gainesville, FL 32611, USA;Department of Chemistry, University of Florida, Gainesville, FL 32611, USA;Department of Physics \& Astronomy, Brigham Young University, Provo, UT 84602, USA;Department of Materials Science \& Engineering, University of Florida, Gainesville, FL 32611, USA;Courant Institute of Mathematical Sciences, New York University, New York 10003, USA;Center for Neural Science, New York University, New York 10003, USA
BoltzNCE: Learning Likelihoods for Boltzmann Generation with Stochastic Interpolants and Noise Contrastive Estimation
CMU-Pitt Computational Biology Dept. of Computational & Systems Biology University of Pittsburgh(卡内基梅隆大学-匹兹堡计算生物学系,计算与系统生物学系,匹兹堡大学);Machine Learning Dept. Dept. of Mathematical Sciences Carnegie Mellon University(机器学习系,数学科学系,卡内基梅隆大学);Dept. of Computational & Systems Biology University of Pittsburgh(计算与系统生物学系,匹兹堡大学)
Benchmarking Stochastic Interpolants for Modeling Physical Systems
Department of Mechanical Engineering(机械工程系);Carnegie Mellon University(卡内基梅隆大学);Department of Geophysical Sciences Committee on Computational and Applied Mathematics(地球科学系计算与应用数学委员会);University of Chicago(芝加哥大学);LMD/IPSL, CNRS, ENS, Université PSL École Polytechnique, Institut Polytechnique de Paris, Sorbonne Université(LMD/IPSL、CNRS、ENS、Université PSL 工程学院、巴黎理工学院、索邦大学);Réseau de Transport d’Électricité (RTE)(电力传输网络(RTE))
Generative Modeling via Kernelized Stochastic Interpolants
CCM, Flatiron Institute, New York, USA(CCM,Flatiron研究所,纽约,美国);Courant Institute of Mathematical Sciences, New York University, New York, USA(数学科学学院,纽约大学,纽约,美国)
Xaurora: Generative Weather Forecasting with Denoising Stochastic Interpolants from a Foundation Model Prior
Vrije Universiteit Amsterdam(阿姆斯特丹自由大学);University of Amsterdam(阿姆斯特丹大学);Delft University of Technology(代尔夫特理工大学);Centrum Wiskunde & Informatica(数学与计算机科学中心)
Stochastic interpolants with data-dependent couplings
Center for Cosmology and Particle Physics, New York University(纽约大学宇宙学与粒子物理中心);Courant Institute of Mathematical Sciences, New York University(纽约大学柯朗数学科学研究所);Center for Data Science, New York University(纽约大学数据科学中心)
Multimarginal generative modeling with stochastic interpolants
Center for Cosmology and Particle Physics(宇宙学与粒子物理中心);New York University(纽约大学);Courant Institute of Mathematical Sciences(库朗数学科学研究所);University of California, Berkeley(加州大学伯克利分校)
Academy of Mathematics and Systems Science, Chinese Academy of Sciences(中国科学院数学与系统科学研究院);University of Chinese Academy of Sciences(中国科学院大学);School of Mathematics and Statistics, Fujian Normal University(福建师范大学数学与统计学学院);Beijing Institute of Technology(北京理工大学)
FlowDAS: A Stochastic Interpolant-based Framework for Data Assimilation
University of Michigan(密歇根大学);Department of Electrical Engineering and Computer Science(电气工程与计算机科学系);Peking University(北京大学);Department of Physics(物理系);National Biomedical Imaging Center(国家生物医学成像中心)
ProGen: Revisiting Probabilistic Spatial-Temporal Time Series Forecasting from a Continuous Generative Perspective Using Stochastic Differential Equations
Hong Kong University of Science and Technology (Guangzhou)(香港科技大学(广州))
Equivariant Neural Simulators for Stochastic Spatiotemporal Dynamics
Data and Artificial Intelligence Cluster, Eindhoven University of Technology(埃因霍温理工大学数据与人工智能集群);Swiss Plasma Center, École Polytechnique Fédérale de Lausanne(洛桑联邦理工学院瑞士等离子体中心)
Anticipating tipping in spatiotemporal systems with machine learning
School of Electrical, Computer, and Energy Engineering, Arizona State University(亚利桑那州立大学电气、计算机与能源工程学院);DEVCOM Army Research Office(DEVCOM陆军研究办公室);Department of Physics, Arizona State University(亚利桑那州立大学物理系)
Integrating Dynamical Systems Modeling with Spatiotemporal scRNA-seq Data Analysis
School of Mathematical Sciences, Peking University(北京大学数学科学学院);Center for Machine Learning Research, Peking University(北京大学机器学习研究中心);Laboratory of Mathematics and Its Applications (LMAM), Peking University(北京大学数学及其应用实验室);Center for Quantitative Biology, Peking University(北京大学定量生物学中心);National Engineering Laboratory for Big Data Analysis and Applications, Peking University(北京大学大数据分析与应用国家工程实验室);AI for Science Institute, Beijing(北京科学智能研究院)