Many real-world systems evolve continuously, yet most machine learning models interpret time series as discrete sequences. Continuous-time approaches instead treat time series as samples from an underlying input path, a formulation that naturally accommodates irregularly sampled or oversampled data. Among these, Neural Controlled Differential Equations (NCDEs) are a maximally expressive class of models that parametrise a vector field using a neural network and evolve their hidden state by solving a dynamical system driven by the input path. NCDEs typically use a non-linear vector field, so their expressive power and continuous-time flexibility come at the cost of a forward pass that is both computationally expensive and inherently sequential, limiting their scalability and practical applicability.
This thesis advances the training and scalability of NCDEs through three complementary contributions. First, building on neural rough differential equations, Log-NCDEs apply the Log-ODE method to efficiently approximate an NCDE's solution during training, improving both computational speed and empirical performance. Second, Linear NCDEs replace the non-linear vector field with a linear one, enabling closed-form solutions and parallel-in-time computation without sacrificing theoretical expressivity. Third, Structured Linear NCDEs use structured linear vector fields to further enhance efficiency while maintaining theoretical expressiveness and empirical performance.
Collectively, these methods reduce the time per training step for an NCDE by up to three orders of magnitude while achieving state-of-the-art performance across diverse time series benchmarks.
Heidelberg Institute for Theoretical Studies(海德堡理论研究所);Mathematical Institute, University of Oxford(牛津大学数学研究所);City University of Hong Kong(香港城市大学);IAR, Karlsruhe Institute of Technology(卡尔斯鲁厄技术大学IAR研究所);Department of Psychiatry, Warneford Hospital(沃内福德医院精神病科)
Universal Time Series Generation with Neural Controlled Differential Equations
Heidelberg Institute for Theoretical Studies(海德堡理论研究所);IAR, Karlsruhe Institute of Technology(卡尔斯鲁厄技术大学IAR部门);Max Planck Institute for Polymer Research(马克斯·普朗克聚合物研究所);IWR, Heidelberg University(海德堡大学IWR部门);Dept. of Computer Science, University of Toronto(多伦多大学计算机科学系);Mathematical Institute, University of Oxford(牛津大学数学研究所);Vector Institute, Toronto, Canada(多伦多向量研究所)
Istituto Italiano di Tecnologia(意大利技术研究院);CNRS(法国国家科学研究中心);CentraleSupélec(中央高等电力学院);Université Paris-Saclay(巴黎-萨克雷大学);SDA Bocconi School of Management(SDA博科尼管理学院);Bocconi University(博科尼大学)
Dynamical Survival Analysis with Controlled Latent States
Inria Paris(法国国家信息与自动化研究所巴黎中心);Centre de Recherche des Cordeliers(科尔德里耶研究所);INSERM(法国国家健康与医学研究院);Université de Paris(巴黎大学);Sorbonne Université(索邦大学);Paris Saclay University(巴黎萨克雷大学);Univ. Paris Cité(巴黎西岱大学)
Bridging Dynamic Factor Models and Neural Controlled Differential Equations for Nowcasting GDP
Korea Financial Telecommunications and Clearings Institute(韩国金融电信与清算院);Yonsei University(延世大学);Korea Advanced Institute of Science and Technology(韩国科学技术院);Korea Institute for International Economic Policy(韩国国际经济政策研究院);Korea Institute of Public Finance(韩国公共财政研究院);Electronics and Telecommunications Research Institute(电子通信研究院)
Acquisition-Independent Deep Learning for Quantitative MRI Parameter Estimation using Neural Controlled Differential Equations
Amsterdam University Medical Center(阿姆斯特丹大学医学中心);Cancer Center Amsterdam(阿姆斯特丹癌症中心);University of Queensland(昆士兰大学);UNSW Sydney(悉尼新南威尔士大学);University of Amsterdam(阿姆斯特丹大学);University of Fribourg(弗里堡大学);University of Bern(伯尔尼大学)