arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2607.05280cs.LG
收藏

神经控制微分方程研究进展

Advances in Neural Controlled Differential Equations

Benjamin Walker

AI总结:

针对神经控制微分方程训练效率低、扩展性差问题,提出Log-NCDE、线性NCDE、结构化线性NCDE三类方法,将训练步耗时降低最高三个数量级,在多时间序列基准上取得SOTA性能。

摘要:

许多现实世界系统是连续演化的,但多数机器学习模型将时间序列解释为离散序列。连续时间方法则将时间序列视为底层输入路径的采样,该范式可天然适配非规则采样或过采样数据。其中神经控制微分方程(NCDEs)是一类表达能力极强的模型,它用神经网络参数化向量场,通过求解由输入路径驱动的动力系统更新隐状态。NCDEs通常采用非线性向量场,其强表达能力与连续时间灵活性的代价是前向传播计算成本高且本质上是串行的,限制了其可扩展性与实际应用价值。本论文通过三项互补性贡献推进了NCDE的训练与可扩展性研究:第一,基于神经粗糙微分方程,Log-NCDE将Log-ODE方法用于训练阶段高效近似NCDE的解,同时提升了计算速度与实证性能。第二,线性NCDE将非线性向量场替换为线性向量场,在不损失理论表达能力的前提下实现了解的闭式求解与时间维度的并行计算。第三,结构化线性NCDE采用结构化线性向量场,在保留理论表达能力与实证性能的同时进一步提升效率。上述方法整体可将NCDE的单训练步耗时降低最高三个数量级,同时在各类时间序列基准上取得当前最优性能。

英文摘要:

Many real-world systems evolve continuously, yet most machine learning models interpret time series as discrete sequences. Continuous-time approaches instead treat time series as samples from an underlying input path, a formulation that naturally accommodates irregularly sampled or oversampled data. Among these, Neural Controlled Differential Equations (NCDEs) are a maximally expressive class of models that parametrise a vector field using a neural network and evolve their hidden state by solving a dynamical system driven by the input path. NCDEs typically use a non-linear vector field, so their expressive power and continuous-time flexibility come at the cost of a forward pass that is both computationally expensive and inherently sequential, limiting their scalability and practical applicability. This thesis advances the training and scalability of NCDEs through three complementary contributions. First, building on neural rough differential equations, Log-NCDEs apply the Log-ODE method to efficiently approximate an NCDE's solution during training, improving both computational speed and empirical performance. Second, Linear NCDEs replace the non-linear vector field with a linear one, enabling closed-form solutions and parallel-in-time computation without sacrificing theoretical expressivity. Third, Structured Linear NCDEs use structured linear vector fields to further enhance efficiency while maintaining theoretical expressiveness and empirical performance. Collectively, these methods reduce the time per training step for an NCDE by up to three orders of magnitude while achieving state-of-the-art performance across diverse time series benchmarks.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑