一种深度学习的打靶公式
A Shooting Formulation of Deep Learning
摘要：
连续深度神经网络可以被视为离散神经网络的深度限制，其动力学类似于常微分方程（ODE）的离散化。虽然已经采取了重要的步骤来实现这种连续制剂的优点，但大多数当前技术不是真正的连续深度，因为它们假定了\textit{identical}层。事实上，现有的作品投入救济无穷维参数空间在学习连续深度神经常微分方程所提出的无数困难。为此，我们引入了一个射击公式，从参数化网络逐层参数化的角度来看，只有一组初始条件描述的最佳网络。对于可扩展性，我们提出了一种新的粒子系综参数化，充分指定的连续深度神经网络的最佳权重轨迹。我们的实验表明，我们的粒子集合射击配方可以实现竞争力的性能，特别是在长期的预测任务。最后，虽然目前的工作受到了连续深度神经网络的启发，但粒子集合射击公式也适用于离散时间网络，并可能导致深度学习参数化的新的肥沃研究领域。
英文摘要：
Continuous-depth neural networks can be viewed as deep limits of discrete neural networks whose dynamics resemble a discretization of an ordinary differential equation (ODE). Although important steps have been taken to realize the advantages of such continuous formulations, most current techniques are not truly continuous-depth as they assume \textit{identical} layers. Indeed, existing works throw into relief the myriad difficulties presented by an infinite-dimensional parameter space in learning a continuous-depth neural ODE. To this end, we introduce a shooting formulation which shifts the perspective from parameterizing a network layer-by-layer to parameterizing over optimal networks described only by a set of initial conditions. For scalability, we propose a novel particle-ensemble parametrization which fully specifies the optimal weight trajectory of the continuous-depth neural network. Our experiments show that our particle-ensemble shooting formulation can achieve competitive performance, especially on long-range forecasting tasks. Finally, though the current work is inspired by continuous-depth neural networks, the particle-ensemble shooting formulation also applies to discrete-time networks and may lead to a new fertile area of research in deep learning parametrization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
