AR-Net: A simple Auto-Regressive Neural Network for time-series
Stanford University(斯坦福大学);Facebook, Inc.(脸书公司)
Oskar Triebe, Nikolay Laptev, Ram Rajagopal
备注:
Building a bridge between traditional statistical time-series models and deep-learning models. Main Topics: Time-Series, Auto-Regression, Neural Networks, Sparsity, Long-Range Dependencies
In this paper we present a new framework for time-series modeling that combines the best of traditional statistical models and neural networks. We focus on time-series with long-range dependencies, needed for monitoring fine granularity data (e.g. minutes, seconds, milliseconds), prevalent in operational use-cases.
Traditional models, such as auto-regression fitted with least squares (Classic-AR) can model time-series with a concise and interpretable model. When dealing with long-range dependencies, Classic-AR models can become intractably slow to fit for large data. Recently, sequence-to-sequence models, such as Recurrent Neural Networks, which were originally intended for natural language processing, have become popular for time-series. However, they can be overly complex for typical time-series data and lack interpretability.
A scalable and interpretable model is needed to bridge the statistical and deep learning-based approaches. As a first step towards this goal, we propose modelling AR-process dynamics using a feed-forward neural network approach, termed AR-Net. We show that AR-Net is as interpretable as Classic-AR but also scales to long-range dependencies.
Our results lead to three major conclusions: First, AR-Net learns identical AR-coefficients as Classic-AR, thus being equally interpretable. Second, the computational complexity with respect to the order of the AR process, is linear for AR-Net as compared to a quadratic for Classic-AR. This makes it possible to model long-range dependencies within fine granularity data. Third, by introducing regularization, AR-Net automatically selects and learns sparse AR-coefficients. This eliminates the need to know the exact order of the AR-process and allows to learn sparse weights for a model with long-range dependencies.
Wasserstein multivariate auto-regressive models for modeling distributional time series
Université Grenoble Alpes, CNRS, Inria, Grenoble INP, LJK(格勒诺布尔阿尔卑斯大学、国家科学研究中心、法国国家信息与自动化研究所、格勒诺布尔INP、LJK研究中心);Institut de Mathématiques de Bordeaux, Université de Bordeaux(波尔多数学研究所、波尔多大学)
Lossless Vocabulary Reduction for Auto-Regressive Language Models
NTT Computer and Data Science Laboratories, NTT Corporation(NTT计算机与数据科学实验室,NTT集团);NTT Human Informatics Laboratories, NTT Corporation(NTT人类信息学实验室,NTT集团)
Benchmarking Long Roll-outs of Auto-regressive Neural Operators for the Compressible Navier-Stokes Equations with Conserved Quantity Correction
Graduate Researcher, Computer Science and Engineering(计算机科学与工程研究生);Postdoctoral Researcher, Mechanical and Aerospace Engineering(机械与航空航天工程博士后);Glenn Chair Professor, Mechanical and Aerospace Engineering(机械与航空航天工程 Glenn 教授);Professor, Computer Science and Engineering(计算机科学与工程教授)
Cactus: Accelerating Auto-Regressive Decoding with Constrained Acceptance Speculative Sampling
Dept. Computing Science & Alberta Machine Intelligence Institute (Amii), University of Alberta(阿尔伯塔大学计算机科学系与阿尔伯塔机器智能研究所);Canada CIFAR AI Chair(加拿大CIFAR人工智能讲席)
XSpecMesh: Quality-Preserving Auto-Regressive Mesh Generation Acceleration via Multi-Head Speculative Decoding
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University(教育部多媒体可信感知与高效计算重点实验室,厦门大学);Shandong Inspur Database Technology Co.,Ltd(山东浪潮数据库技术有限公司)
Non-Stationary Bandits with Auto-Regressive Temporal Dependency
Operations Research Center, Massachusetts Institute of Technology(麻省理工学院运营研究中心);Sloan School of Management, Massachusetts Institute of Technology(麻省理工学院斯隆管理学院);IBM Research(IBM研究院)
Brain dynamics via Cumulative Auto-Regressive Self-Attention
Tri-institutional Center for Translational Research in Neuroimaging and Data Science (TReNDS)(神经影像与数据科学转化研究三机构中心(TReNDS));Georgia State University(佐治亚州立大学);Georgia Tech(佐治亚理工学院);Emory(埃默里大学)
X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Models
Shanghai Jiao Tong University(上海交通大学);Shanghai AI Laboratory(上海人工智能实验室);Tsinghua University(清华大学);The Chinese University of Hong Kong(香港中文大学);CPII under InnoHK(创新香港科技促进会);MThreads AI