arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2103.15213cs.LG
收藏

基于时间核的连续时间信息深度学习方法

A Temporal Kernel Approach for Deep Learning with Continuous-time Information

Walmart Labs(沃尔玛实验室);Instacart

Da Xu, Chuanwei Ruan, Evren Korpeoglu, Sushant Kumar, Kannan Achan

摘要:

顺序深度学习模型,如RNN、因果CNN和注意力机制,不容易消耗连续时间信息。如我们所示,离散化时态数据会导致不一致,即使是简单的连续时间过程。当前的方法通常以启发式方式处理时间,以与现有的深度学习架构和实现一致。在本文中,我们提供了一种使用深度学习工具来表征连续时间系统的原则性方法。值得注意的是,所提出的方法适用于所有主要的深度学习架构,并且只需要对实现进行很少的修改。关键的见解是通过将神经网络与时间内核组合来表示连续时间系统,其中我们从最近对高斯过程和神经切线核的深度学习的理解中获得了直觉。为了表示时间核,我们引入了随机特征方法,并将核学习问题转换为重新参数化下的谱密度估计。我们进一步证明了收敛性和一致性的结果,即使时间内核是非平稳的,和谱密度是错误的。模拟和真实数据实验表明,我们的时间内核方法在广泛的设置经验的有效性。

英文摘要:

Sequential deep learning models such as RNN, causal CNN and attention mechanism do not readily consume continuous-time information. Discretizing the temporal data, as we show, causes inconsistency even for simple continuous-time processes. Current approaches often handle time in a heuristic manner to be consistent with the existing deep learning architectures and implementations. In this paper, we provide a principled way to characterize continuous-time systems using deep learning tools. Notably, the proposed approach applies to all the major deep learning architectures and requires little modifications to the implementation. The critical insight is to represent the continuous-time system by composing neural networks with a temporal kernel, where we gain our intuition from the recent advancements in understanding deep learning with Gaussian process and neural tangent kernel. To represent the temporal kernel, we introduce the random feature approach and convert the kernel learning problem to spectral density estimation under reparameterization. We further prove the convergence and consistency results even when the temporal kernel is non-stationary, and the spectral density is misspecified. The simulations and real-data experiments demonstrate the empirical effectiveness of our temporal kernel approach in a broad range of settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑