Neural Autoregressive Distribution Estimators (NADEs) have recently been shown as successful alternatives for modeling high dimensional multimodal distributions. One issue associated with NADEs is that they rely on a particular order of factorization for $P(\mathbf{x})$. This issue has been recently addressed by a variant of NADE called Orderless NADEs and its deeper version, Deep Orderless NADE. Orderless NADEs are trained based on a criterion that stochastically maximizes $P(\mathbf{x})$ with all possible orders of factorizations. Unfortunately, ancestral sampling from deep NADE is very expensive, corresponding to running through a neural net separately predicting each of the visible variables given some others. This work makes a connection between this criterion and the training criterion for Generative Stochastic Networks (GSNs). It shows that training NADEs in this way also trains a GSN, which defines a Markov chain associated with the NADE model. Based on this connection, we show an alternative way to sample from a trained Orderless NADE that allows to trade-off computing time and quality of the samples: a 3 to 10-fold speedup (taking into account the waste due to correlations between consecutive samples of the chain) can be obtained without noticeably reducing the quality of the samples. This is achieved using a novel sampling procedure for GSNs called annealed GSN sampling, similar to tempering methods that combines fast mixing (obtained thanks to steps at high noise levels) with accurate samples (obtained thanks to steps at low noise levels).
The Gaussian equivalence of generative models for learning with shallow neural networks
International School of Advanced Studies (SISSA)（国际高等研究学院（SISSA））；Duke University（杜克大学）；Université PSL（巴黎文理研究大学）；Sorbonne Université（索邦大学）；Université Paris-Diderot（巴黎第七大学）
From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training
California Institute of Technology（加州理工学院）；NVIDIA（英伟达）；Zuse Institute Berlin（柏林泽尼克研究所）；dida Datenschmiede GmbH（dida数据隐私公司）；Jagiellonian University（雅盖隆大学）；Mila, Université de Montréal（蒙特利尔大学机器学习研究所）；University of Edinburgh（爱丁堡大学）；CIFAR Fellow, Learning in Machines and Brains（CIFAR Fellow, 机器学习与大脑学习）
Equivalence Between Wasserstein and Value-Aware Loss for Model-based Reinforcement Learning
Brown University（布朗大学）；Cornell Tech（康奈尔科技校区）
生成模型的学习是基于模型的强化学习的关键组成部分。虽然在表格设置中学习一个好的模型是一项简单的任务，但在近似设置中学习一个有用的模型是具有挑战性的。在这种情况下，一个重要的问题是用于模型学习的损失函数，因为改变损失函数可以对规划的有效性产生显著影响。最近，Farahmand et al.（2017）提出了一种值感知模型学习（VAML）目标，该目标在模型学习期间捕获价值函数的结构。使用Asadi et al.（2018）的工具，我们证明了最小化VAML目标实际上等同于最小化Wasserstein度量。这种等价性加深了我们对值感知模型的理解，也为Wasserstein在基于模型的强化学习中的应用奠定了理论基础。
New Equivalences Between Interpolation and SVMs: Kernels and Structured Features
School of Electrical and Computer Engineering, Georgia Institute of Technology（佐治亚理工学院电子与计算机工程系）；Institute of Mathematics, EPFL（瑞士联邦理工学院数学学院）；School of Industrial & Systems Engineering, Georgia Institute of Technology（佐治亚理工学院工业与系统工程系）
State Key Laboratory of General Artificial Intelligence, Peking University, Beijing, China（一般人工智能国家重点实验室，北京大学，北京，中国）；Huazhong University of Science and Technology, Wuhan, China（华中科技大学，武汉，中国）；Microsoft Research Asia, Beijing, China（微软亚洲研究院，北京，中国）；Zhongguancun Academy, Beijing, China（中关村学院，北京，中国）
Modelling the influence of data structure on learning in neural networks: the hidden manifold model
Laboratoire de Physique de l’Ecole Normale Supérieure（巴黎高等师范学院物理实验室）；Université PSL（巴黎文理研究大学）；CNRS（法国国家科学研究中心）；Sorbonne Université（索邦大学）；Université Paris-Diderot（巴黎狄德罗大学）；Institut de Physique Théorique（理论物理研究所）；CEA（法国原子能和替代能源委员会）；Université Paris-Saclay（巴黎-萨克雷大学）
