arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2501.18871cs.LGstat.ML
收藏

Neural SDEs作为连续域序列建模的统一方法

Neural SDEs as a Unified Approach to Continuous-Domain Sequence Modeling

Shanghai QiZhi Institute(上海智己研究院);University of Pennsylvania(宾夕法尼亚大学)

Macheng Shen, Chen Cheng

AI总结:

提出基于Neural SDE的连续序列建模方法,将时间序列视为连续动力系统的离散样本,推导最大似然目标与无模拟训练方案,在具身与生成AI任务中验证有效性。

摘要:

受微分方程在多个科学与工程领域中广泛用于建模连续动态的启发,我们提出了一种新颖且直观的连续序列建模方法。该方法将时间序列数据解释为来自底层连续动力系统的离散样本,并使用Neural Stochastic Differential Equation(Neural SDE,神经随机微分方程)对其时间演化进行建模,其中流(漂移)项和扩散项均由神经网络参数化。我们推导出了一个原则性的最大似然目标函数以及一种无模拟方案,以高效训练Neural SDE模型。我们通过在具身AI和生成式AI的序列建模任务上的实验,展示了该方法的通用性。值得注意的是,据我们所知,这是首个证明基于SDE的连续时间建模在此类复杂场景中同样表现优异的工作,我们希望本研究能为SDE模型在高维和时间复杂领域的研究开辟新途径。

英文摘要:

Inspired by the ubiquitous use of differential equations to model continuous dynamics across diverse scientific and engineering domains, we propose a novel and intuitive approach to continuous sequence modeling. Our method interprets time-series data as \textit{discrete samples from an underlying continuous dynamical system}, and models its time evolution using Neural Stochastic Differential Equation (Neural SDE), where both the flow (drift) and diffusion terms are parameterized by neural networks. We derive a principled maximum likelihood objective and a \textit{simulation-free} scheme for efficient training of our Neural SDE model. We demonstrate the versatility of our approach through experiments on sequence modeling tasks across both embodied and generative AI. Notably, to the best of our knowledge, this is the first work to show that SDE-based continuous-time modeling also excels in such complex scenarios, and we hope that our work opens up new avenues for research of SDE models in high-dimensional and temporally intricate domains.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑