我们提出了一种名为Time Series Representation Model (TSRM)的时间特征编码架构，用于多变量时间序列预测和插补。该架构以基于CNN的表示层为核心，每层专注于独立的表示学习任务并旨在捕获多样的时间模式，随后是基于注意力的特征提取层和用于聚合提取特征的合并层。该架构从根本上基于受Transformer编码器启发的配置，并以自注意力机制为核心。在我们经验性评估所考虑的7个既定基准数据集的大多数数据集上，TSRM架构在预测和插补任务中均优于现有最先进的方法。同时，它以可学习参数的形式显著降低了复杂性。源代码可在https://github.com/RobertLeppich/TSRM获取。
英文摘要：
We introduce a temporal feature encoding architecture called Time Series Representation Model (TSRM) for multivariate time series forecasting and imputation. The architecture is structured around CNN-based representation layers, each dedicated to an independent representation learning task and designed to capture diverse temporal patterns, followed by an attention-based feature extraction layer and a merge layer, designed to aggregate extracted features. The architecture is fundamentally based on a configuration that is inspired by a Transformer encoder, with self-attention mechanisms at its core. The TSRM architecture outperforms state-of-the-art approaches on most of the seven established benchmark datasets considered in our empirical evaluation for both forecasting and imputation tasks. At the same time, it significantly reduces complexity in the form of learnable parameters. The source code is available at https://github.com/RobertLeppich/TSRM.
Self-Supervised Learning of Time Series Representation via Diffusion Process and Imputation-Interpolation-Forecasting Mask
Motherbrain, EQT Group（Motherbrain、EQT集团）；KTH Royal Institute of Technology（皇家理工学院）；Mitsubishi UFJ Trust Investment Technology Institute（三菱UFJ信托投资技术研究所）；Technical University of Denmark（丹麦技术大学）
Is Precise Recovery Necessary? A Task-Oriented Imputation Approach for Time Series Forecasting on Variable Subset
University of Macau（澳门大学）；Northeast Normal University（东北师范大学）；Computer Network Information Center, Chinese Academy of Sciences（中国科学院计算机网络信息中心）；University of Oxford（牛津大学）；Dalian Maritime University（大连海事大学）
Nonparametric End-to-End Probabilistic Forecasting of Distributed Generation Outputs Considering Missing Data Imputation
Guangzhou Power Supply Bureau Guangdong Power Grid Co., Ltd.（广东电网有限责任公司广州供电局）；Tsinghua University（清华大学）；Tsinghua-Berkeley Shenzhen Institute (TBSI)（清华-伯克利深圳学院）
A Survey on Graph Neural Networks for Time Series: Forecasting, Classification, Imputation, and Anomaly Detection
Griffith University（格里菲斯大学）；Monash University（莫纳什大学）；Squirrel AI（松鼠AI）；IDSIA（瑞士IDSIA人工智能实验室）；Università della Svizzera italiana（意大利语瑞士大学）；Politecnico di Milano（米兰理工大学）；The Chinese University of Hong Kong（香港中文大学）
Comparative Analysis of Modern Machine Learning Models for Retail Sales Forecasting
Faculty of Electrical Engineering and Computing, University of Zagreb（Zagreb大学电气工程与计算学院）；It From Bit d.o.o.（It From Bit公司）；Babson College（巴布森学院）；mStart Plus d.o.o.（mStart Plus公司）
Time series are all around in real-world applications. However, unexpected accidents for example broken sensors or missing of the signals will cause missing values in time series, making the data hard to be utilized. It then does harm to the downstream applications such as traditional classification or regression, sequential data integration and forecasting tasks, thus raising the demand for data imputation. Currently, time series data imputation is a well-studied problem with different categories of methods. However, these works rarely take the temporal relations among the observations and tr…
Probabilistic Computing Project Massachusetts Institute of Technology（麻省理工学院概率计算项目）
This article proposes a Bayesian nonparametric method for forecasting, imputation, and clustering in sparsely observed, multivariate time series data. The method is appropriate for jointly modeling hundreds of time series with widely varying, non-stationary dynamics. Given a collection of $N$ time series, the Bayesian model first partitions them into independent clusters using a Chinese restaurant process prior. Within a cluster, all time series are modeled jointly using a novel "temporally-reweighted" extension of the Chinese restaurant process mixture. Markov chain Monte Carlo techniques are…
KDD CUP 2022 proposes a time-series forecasting task on spatial dynamic wind power dataset, in which the participants are required to predict the future generation given the historical context factors. The evaluation metrics contain RMSE and MAE. This paper describes the solution of Team 88VIP, which mainly comprises two types of models: a gradient boosting decision tree to memorize the basic data patterns and a recurrent neural network to capture the deep and latent probabilistic transitions. Ensembling these models contributes to tackle the fluctuation of wind power, and training submodels t…
Marginalize, Rather than Impute: Probabilistic Wind Power Forecasting with Incomplete Data
School of Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电气工程学院）；Dyson School of Design Engineering, Imperial College London（帝国理工学院设计工程学院）；Department of Technology, Management and Economics, Technical University of Denmark（丹麦技术大学技术、管理与经济系）；Centre for Energy Research – CoRE, Aarhus University（奥胡斯大学能源研究中心）
Short-term traffic forecasting based on deep learning methods, especially recurrent neural networks (RNN), has received much attention in recent years. However, the potential of RNN-based models in traffic forecasting has not yet been fully exploited in terms of the predictive power of spatial-temporal data and the capability of handling missing data. In this paper, we focus on RNN-based models and attempt to reformulate the way to incorporate RNN and its variants into traffic prediction models. A stacked bidirectional and unidirectional LSTM network architecture (SBU-LSTM) is proposed to assi…
Traffic forecasting has attracted widespread attention recently. In reality, traffic data usually contains missing values due to sensor or communication errors. The Spatio-temporal feature in traffic data brings more challenges for processing such missing values, for which the classic techniques (e.g., data imputations) are limited: 1) in temporal axis, the values can be randomly or consecutively missing; 2) in spatial axis, the missing values can happen on one single sensor or on multiple sensors simultaneously. Recent models powered by Graph Neural Networks achieved satisfying performance on…
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；Meta；IBM Research（IBM研究院）
Time series data appears in a variety of applications such as smart transportation and environmental monitoring. One of the fundamental problems for time series analysis is time series forecasting. Despite the success of recent deep time series forecasting methods, they require sufficient observation of historical values to make accurate forecasting. In other words, the ratio of the output length (or forecasting horizon) to the sum of the input and output lengths should be low enough (e.g., 0.3). As the ratio increases (e.g., to 0.8), the uncertainty for the forecasting accuracy increases sign…
Carnegie Mellon University（卡内基梅隆大学）；AWS AI Labs（AWS人工智能实验室）
In this work, we tackle two widespread challenges in real applications for time-series forecasting that have been largely understudied: distribution shifts and missing data. We propose SpectraNet, a novel multivariate time-series forecasting model that dynamically infers a latent space spectral decomposition to capture current temporal dynamics and correlations on the recent observed history. A Convolution Neural Network maps the learned representation by sequentially mixing its components and refining the output. Our proposed approach can simultaneously produce forecasts and interpolate past …
Learning Data-Driven Uncertainty Set Partitions for Robust and Adaptive Energy Forecasting with Missing Data
Department of Electrical and Electronic Engineering, Imperial College London（帝国理工学院电子与电气工程系）；Center for Processes, Renewable Energy and Energy Systems (PERSEE), Mines Paris, PSL University（进程、可再生能源与能源系统中心（PERSEE），巴黎 Mines，PSL 大学）
CoIFNet: A Unified Framework for Multivariate Time Series Forecasting with Missing Values
School of Computing and Artificial Intelligence, Southwest Jiaotong University（计算机与人工智能学院，西南交通大学）；School of Mathematics, Southwest Jiaotong University（数学学院，西南交通大学）；Institute Carbon Neutrality, Tsinghua University（碳中和研究院，清华大学）；White Rose Grid e-Science Centre, University of Leeds（白玫瑰网格e科学中心，利兹大学）
Time series prediction has been a long-standing research topic and an essential application in many domains. Modern time series collected from sensor networks (e.g., energy consumption and traffic flow) are often large-scale and incomplete with considerable corruption and missing values, making it difficult to perform accurate predictions. In this paper, we propose a low-rank autoregressive tensor completion (LATC) framework to model multivariate time series data. The key of LATC is to transform the original multivariate time series matrix (e.g., sensor$\times$time point) to a third-order tens…
Universidad Nacional de Colombia（哥伦比亚国立大学）；University of Florida（佛罗里达大学）
Accurate mechanisms for forecasting solar irradiance and insolation provide important information for the planning of renewable energy and agriculture projects as well as for environmental and socio-economical studies. This research introduces a pipeline for the one-day ahead forecasting of solar irradiance and insolation that only requires solar irradiance historical data for training. Furthermore, our approach is able to deal with missing data since it includes a data imputation state. In the prediction stage, we consider four data-driven approaches: Autoregressive Integrated Moving Average …
Universidad Carlos III de Madrid（马德里卡洛斯三世大学）；University of Eastern Finland（东芬兰大学）
Machine learning techniques typically applied to dementia forecasting lack in their capabilities to jointly learn several tasks, handle time dependent heterogeneous data and missing values. In this paper, we propose a framework using the recently presented SSHIBA model for jointly learning different tasks on longitudinal data with missing values. The method uses Bayesian variational inference to impute missing values and combine information of several views. This way, we can combine different data-views from different time-points in a common latent space and learn the relations between each ti…
A Lightweight DL Model for Smart Grid Power Forecasting with Feature and Resolution Mismatch
Center for Automotive Research (CAR), The Ohio State University（汽车研究中心（CAR），俄亥俄州立大学）；Informatics Institute, Istanbul Technical University（信息学院，伊斯坦布尔技术大学）；Computer Engineering Department, Konkuk University（计算机工程系，韩国康 kuk 大学）；Computer Science and Information Technology Department, Maulana Azad National Urdu University（计算机科学与信息技术系，莫拉尔·阿扎德国家乌尔都语大学）
Clemson University（克莱姆森大学）；West Virginia University（西弗吉尼亚大学）
The prediction of high-resolution hourly traffic volumes of a given roadway is essential for transportation planning. Traditionally, Automatic Traffic Recorders (ATR) are used to collect this hourly volume data. These large datasets are time series data characterized by long-term temporal dependencies and missing values. Regarding the temporal dependencies, all roadways are characterized by seasonal variations that can be weekly, monthly or yearly, depending on the cause of the variation. Regarding the missing data in a time-series sequence, traditional time series forecasting models perform p…
TimeAutoDiff: A Unified Framework for Generation, Imputation, Forecasting, and Time-Varying Metadata Conditioning of Heterogeneous Time Series Tabular Data
University of Washington（华盛顿大学）；Beihang University（北京航空航天大学）
Traffic forecasting is a classical task for traffic management and it plays an important role in intelligent transportation systems. However, since traffic data are mostly collected by traffic sensors or probe vehicles, sensor failures and the lack of probe vehicles will inevitably result in missing values in the collected raw data for some specific links in the traffic network. Although missing values can be imputed, existing data imputation methods normally need long-term historical traffic state data. As for short-term traffic forecasting, especially under edge computing and online predicti…
DAM: Towards A Foundation Model for Time Series Forecasting
School of Computer Science, University of St Andrews（圣安德鲁斯大学计算机学院）；Systems Infrastructure Research（系统基础设施研究机构）；Edinburgh Research Centre（爱丁堡研究中心）；Central Software Institute（中央软件研究院）；Huawei（华为）
In this work we propose for the first time a transformer-based framework for unsupervised representation learning of multivariate time series. Pre-trained models can be potentially used for downstream tasks such as regression and classification, forecasting and missing value imputation. By evaluating our models on several benchmark datasets for multivariate time series regression and classification, we show that not only does our modeling approach represent the most successful method employing unsupervised learning of multivariate time series presented to date, but also that it exceeds the cur…
PyPOTS Research（PyPOTS研究机构）；HKUST（香港科技大学）；King’s College London（伦敦国王学院）；University of Oxford（牛津大学）；Sichuan University（四川大学）；University of Washington（华盛顿大学）；GaTech（佐治亚理工学院）；HKUST(GZ)（香港科技大学（广州））；Northwest University（西北大学）；Squirrel AI（松鼠AI）
TriQXNet: Forecasting Dst Index from Solar Wind Data Using an Interpretable Parallel Classical-Quantum Framework with Uncertainty Quantification
Department of Computer Science, Viterbi School of Engineering, University of Southern California（计算机科学系，维特比工程学院，南加州大学）；Department of Computer Science, American International University-Bangladesh（计算机科学系，美国国际大学-孟加拉国）；Department of Computer Science & Engineering, Techno International New Town（计算机科学与工程系，Techno International New Town）；Department of Biomedical Engineering, University of Reading（生物医学工程系，阅读大学）
There has been increased interest in missing sensor data imputation, which is ubiquitous in the field of structural health monitoring (SHM) due to discontinuous sensing caused by sensor malfunction. To address this fundamental issue, this paper presents an incremental Bayesian tensor learning method for reconstruction of spatiotemporal missing data in SHM and forecasting of structural response. In particular, a spatiotemporal tensor is first constructed followed by Bayesian tensor factorization that extracts latent features for missing data imputation. To enable structural response forecasting…
TCS Research（塔塔咨询服务公司研究院）；IIT Madras Research Park（印度理工学院马德拉斯分校研究园）
Missing data scenarios are very common in ML applications in general and time-series/sequence applications are no exceptions. This paper pertains to a novel Recurrent Neural Network (RNN) based solution for sequence prediction under missing data. Our method is distinct from all existing approaches. It tries to encode the missingness patterns in the data directly without trying to impute data either before or during model building. Our encoding is lossless and achieves compression. It can be employed for both sequence classification and forecasting. We focus on forecasting here in a general con…
TimeOmni-1: Incentivizing Complex Reasoning with Time Series in Large Language Models
Griffith University（格里菲斯大学）；Zhejiang University（浙江大学）；NVIDIA（英伟达）；Squirrel Ai Learning；University of Palermo（帕尔米奥大学）；Norwegian University of Science and Technology（挪威科学技术大学）
