Consider a multi-variate time series $(X_t)_{t=0}^{T}$ where $X_t \in \mathbb{R}^d$ which may represent spike train responses for multiple neurons in a brain, crime event data across multiple regions, and many others. An important challenge associated with these time series models is to estimate an influence network between the $d$ variables, especially when the number of variables $d$ is large meaning we are in the high-dimensional setting. Prior work has focused on parametric vector auto-regressive models. However, parametric approaches are somewhat restrictive in practice. In this paper, we use the non-parametric sparse additive model (SpAM) framework to address this challenge. Using a combination of $β$ and $ϕ$-mixing properties of Markov chains and empirical process techniques for reproducing kernel Hilbert spaces (RKHSs), we provide upper bounds on mean-squared error in terms of the sparsity $s$, logarithm of the dimension $\log d$, number of time points $T$, and the smoothness of the RKHSs. Our rates are sharp up to logarithm factors in many cases. We also provide numerical experiments that support our theoretical results and display potential advantages of using our non-parametric SpAM framework for a Chicago crime dataset.
Lossless Vocabulary Reduction for Auto-Regressive Language Models
NTT Computer and Data Science Laboratories, NTT Corporation(NTT计算机与数据科学实验室,NTT集团);NTT Human Informatics Laboratories, NTT Corporation(NTT人类信息学实验室,NTT集团)
Wasserstein multivariate auto-regressive models for modeling distributional time series
Université Grenoble Alpes, CNRS, Inria, Grenoble INP, LJK(格勒诺布尔阿尔卑斯大学、国家科学研究中心、法国国家信息与自动化研究所、格勒诺布尔INP、LJK研究中心);Institut de Mathématiques de Bordeaux, Université de Bordeaux(波尔多数学研究所、波尔多大学)
Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling
Dept. Computing Science & Alberta Machine Intelligence Institute (Amii), University of Alberta(阿尔伯塔大学计算机科学系与阿尔伯塔机器智能研究所);Canada CIFAR AI Chair(加拿大CIFAR人工智能讲席)