arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2408.08055cs.LGcs.AI
收藏

DeNOTS: 稳定的深度神经微分方程用于时间序列

DeNOTS: Stable Deep Neural ODEs for Time Series

Applied AI Institute(应用人工智能研究所);Innotech(Innotech公司)

Ilya Kuleshov, Evgenia Romanenkova, Vladislav Zhuzhel, Galina Boeva, Evgeni Vorsin, Alexey Zaytsev

AI总结:

DeNOTS通过扩展积分时间范围和负反馈机制,提升了神经微分方程在时间序列建模中的稳定性与鲁棒性,实现了20%的性能提升。

摘要:

神经CDEs提供了一种自然的方法来处理不规则时间序列的时间演变。函数评估次数(NFE)是这些系统自然的深度类比(传统神经网络的层数)。通常通过求解器误差容限来调节:较低的容限意味着更高的数值精度,需要更多的积分步骤。然而,降低容限并不能充分增加模型的表达能力。我们提出了一种简单而有效的替代方法:通过扩展积分时间范围来增加NFE并“加深”模型。增加积分区间会导致传统向量场的不可控增长,因此我们还提出了一种通过负反馈(NF)来稳定动态的方法。它确保了可证明的稳定性而不限制灵活性。它还意味着鲁棒性:我们利用高斯过程理论为神经ODE风险提供了理论界限。在四个公开数据集上的实验表明,我们的方法DeNOTS在现有方法——包括最近的神经RDEs和状态空间模型——中表现更优,实现了高达20%的改进。DeNOTS结合了表达力、稳定性和鲁棒性,使在连续时间域中的建模变得可靠。

英文摘要:

Neural CDEs provide a natural way to process the temporal evolution of irregular time series. The number of function evaluations (NFE) is these systems' natural analog of depth (the number of layers in traditional neural networks). It is usually regulated via solver error tolerance: lower tolerance means higher numerical precision, requiring more integration steps. However, lowering tolerances does not adequately increase the models' expressiveness. We propose a simple yet effective alternative: scaling the integration time horizon to increase NFEs and "deepen`` the model. Increasing the integration interval causes uncontrollable growth in conventional vector fields, so we also propose a way to stabilize the dynamics via Negative Feedback (NF). It ensures provable stability without constraining flexibility. It also implies robustness: we provide theoretical bounds for Neural ODE risk using Gaussian process theory. Experiments on four open datasets demonstrate that our method, DeNOTS, outperforms existing approaches~ -- ~including recent Neural RDEs and state space models,~ -- ~achieving up to $20\%$ improvement in metrics. DeNOTS combines expressiveness, stability, and robustness, enabling reliable modelling in continuous-time domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑