Transformer-based models have emerged as powerful tools for multivariate time series forecasting (MTSF). However, existing Transformer models often fall short of capturing both intricate dependencies across variate and temporal dimensions in MTS data. Some recent models are proposed to separately capture variate and temporal dependencies through either two sequential or parallel attention mechanisms. However, these methods cannot directly and explicitly learn the intricate inter-series and intra-series dependencies. In this work, we first demonstrate that these dependencies are very important as they usually exist in real-world data. To directly model these dependencies, we propose a transformer-based model UniTST containing a unified attention mechanism on the flattened patch tokens. Additionally, we add a dispatcher module which reduces the complexity and makes the model feasible for a potentially large number of variates. Although our proposed model employs a simple architecture, it offers compelling performance as shown in our extensive experiments on several datasets for time series forecasting.
Robust Multivariate Time Series Forecasting against Intra- and Inter-Series Transitional Shift
School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）；Tongji University（同济大学）；School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Data Science Institute, University of Technology Sydney（悉尼科技大学数据科学研究所）；School of Computing, Macquarie University（麦考瑞大学计算学院）
Deep Coupling Network For Multivariate Time Series Forecasting
School of Computer Science, Beijing Institute of Technology（北京理工大学计算机学院）；School of Electronic and Information Engineering, Tongji University（同济大学电子与信息工程学院）；School of Medical Technology, Beijing Institute of Technology（北京理工大学医学技术学院）；Faculty of Engineering and Information Technology, University of Technology Sydney（悉尼科技大学工程与信息技术学院）；School of Computer Science and Information Engineering, HeFei University of Technology（合肥工业大学计算机与信息工程学院）
Revitalizing Multivariate Time Series Forecasting: Learnable Decomposition with Inter-Series Dependencies and Intra-Series Variations Modeling
University of Electronic Science and Technology of China（电子科技大学）；The Hong Kong Polytechnic University（香港理工大学）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；University of Cambridge（剑桥大学）
Radiant First Research Limited（Radiant First研究有限公司）；The University of Hong Kong（香港大学）；Koi Investment Partners International Limited（Koi国际投资伙伴有限公司）
A reliable and efficient representation of multivariate time series is crucial in various downstream machine learning tasks. In multivariate time series forecasting, each variable depends on its historical values and there are inter-dependencies among variables as well. Models have to be designed to capture both intra- and inter-relationships among the time series. To move towards this goal, we propose the Time Series Attention Transformer (TSAT) for multivariate time series representation learning. Using TSAT, we represent both temporal information and inter-dependencies of multivariate time …
Multivariate time series (MTS) forecasting has attracted much attention in many intelligent applications. It is not a trivial task, as we need to consider both intra-variable dependencies and inter-variable dependencies. However, existing works are designed for specific scenarios, and require much domain knowledge and expert efforts, which is difficult to transfer between different scenarios. In this paper, we propose a scale-aware neural architecture search framework for MTS forecasting (SNAS4MTF). A multi-scale decomposition module transforms raw time series into multi-scale sub-series, whic…
Anomalies in univariate time series often refer to abnormal values and deviations from the temporal patterns from majority of historical observations. In multivariate time series, anomalies also refer to abnormal changes in the inter-series relationship, such as correlation, over time. Existing studies have been able to model such inter-series relationships through graph neural networks. However, most works settle on learning a static graph globally or within a context window to assist a time series forecasting task or a reconstruction task, whose objective is not tailored to explicitly detect…
Structured Latent Space Modeling over Multi-Scale Temporal Patches for Multivariate Time Series Forecasting
School of Computing and Data Science, The University of Hong Kong（香港大学计算与数据科学学院）；Innovation Engineering College, Macau University of Science and Technology（澳门科技大学创新工程学院）
National University of Singapore（新加坡国立大学）；Nanyang Technological University（南洋理工大学）；Peking University（北京大学）
Learning information-rich and generalizable representations effectively from unlabeled multivariate cardiac signals to identify abnormal heart rhythms (cardiac arrhythmias) is valuable in real-world clinical settings but often challenging due to its complex temporal dynamics. Cardiac arrhythmias can vary significantly in temporal patterns even for the same patient ($i.e.$, intra subject difference). Meanwhile, the same type of cardiac arrhythmia can show different temporal patterns among different patients due to different cardiac structures ($i.e.$, inter subject difference). In this paper, w…
FourierGNN: Rethinking Multivariate Time Series Forecasting from a Pure Graph Perspective
Beijing Institute of Technology（北京理工大学）；Tongji University（同济大学）；University of Oxford（牛津大学）；University of Macau（澳门大学）；HeFei University of Technology（合肥工业大学）；Macquarie University（麦考瑞大学）
Pay Attention to Evolution: Time Series Forecasting with Deep Graph-Evolution Learning
University of Sao Paulo（圣保罗大学）；Dalhousie University（达尔豪斯大学）；Peking University（北京大学）；Polish Academy of Sciences（波兰科学院）；Université Grenoble Alpes（格勒诺布尔阿尔卑斯大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
OrderFusion: Encoding Orderbook for End-to-End Probabilistic Intraday Electricity Price Forecasting
Delft University of Technology（代尔夫特理工大学）；Austrian Institute of Technology（奥地利技术研究院）；RWTH Aachen（亚琛工业大学）；London Business School（伦敦商学院）；University of Oxford（牛津大学）；Squirrel AI（Squirrel AI公司）；Tsinghua University（清华大学）
Forschungszentrum Jülich GmbH（于利希研究中心）；RWTH Aachen University（亚琛工业大学）；University of Cologne（科隆大学）；JARA-ENERGY（于利希-亚琛研究联盟能源部）
Electricity is traded on various markets with different time horizons and regulations. Short-term intraday trading becomes increasingly important due to the higher penetration of renewables. In Germany, the intraday electricity price typically fluctuates around the day-ahead price of the European Power EXchange (EPEX) spot markets in a distinct hourly pattern. This work proposes a probabilistic modeling approach that models the intraday price difference to the day-ahead contracts. The model captures the emerging hourly pattern by considering the four 15 min intervals in each day-ahead price in…
Hellebore Capital Management（Hellebore资本管理公司）；Ecole Polytechnique（巴黎综合理工学院）
This paper presents a new methodology for clustering multivariate time series leveraging optimal transport between copulas. Copulas are used to encode both (i) intra-dependence of a multivariate time series, and (ii) inter-dependence between two time series. Then, optimal copula transport allows us to define two distances between multivariate time series: (i) one for measuring intra-dependence dissimilarity, (ii) another one for measuring inter-dependence dissimilarity based on a new multivariate dependence coefficient which is robust to noise, deterministic, and which can target specified dep…
Multivariate time-series forecasting plays a crucial role in many real-world applications. It is a challenging problem as one needs to consider both intra-series temporal correlations and inter-series correlations simultaneously. Recently, there have been multiple works trying to capture both correlations, but most, if not all of them only capture temporal correlations in the time domain and resort to pre-defined priors as inter-series relationships. In this paper, we propose Spectral Temporal Graph Neural Network (StemGNN) to further improve the accuracy of multivariate time-series forecastin…
Static and multivariate-temporal attentive fusion transformer for readmission risk prediction
School of Computer and Artificial Intelligence, Zhengzhou University（郑州大学计算机与人工智能学院）；Cooperative Innovation Center of Internet Healthcare, Zhengzhou University（郑州大学互联网医疗协同创新中心）；Fuwai Hospital, National Center for Cardiovascular Diseases, Chinese Academy of Medical Sciences and Peking Union Medical College（中国医学科学院北京协和医学院国家心血管病中心阜外医院）
McKelvey School of Engineering, Washington University in St. Louis（圣路易斯华盛顿大学麦凯尔维工程学院）；School of Medicine, Washington University in St. Louis（圣路易斯华盛顿大学医学院）
Major postoperative complications are devastating to surgical patients. Some of these complications are potentially preventable via early predictions based on intraoperative data. However, intraoperative data comprise long and fine-grained multivariate time series, prohibiting the effective learning of accurate models. The large gaps associated with clinical events and protocols are usually ignored. Moreover, deep models generally lack transparency. Nevertheless, the interpretability is crucial to assist clinicians in planning for and delivering postoperative care and timely interventions. Tow…
Maastricht University（马斯特里赫特大学）；University of Amsterdam（阿姆斯特丹大学）
In the field of psychopathology, Ecological Momentary Assessment (EMA) methodological advancements have offered new opportunities to collect time-intensive, repeated and intra-individual measurements. This way, a large amount of data has become available, providing the means for further exploring mental disorders. Consequently, advanced machine learning (ML) methods are needed to understand data characteristics and uncover hidden and meaningful relationships regarding the underlying complex psychological processes. Among other uses, ML facilitates the identification of similar patterns in data…
Seagate Technology（希捷科技）；Florida Atlantic University（佛罗里达大西洋大学）；University of Chicago（芝加哥大学）；University of Cincinnati（辛辛那提大学）；Stanford University（斯坦福大学）；Case Western Reserve University（凯斯西储大学）
In the era of big data, data-driven based classification has become an essential method in smart manufacturing to guide production and optimize inspection. The industrial data obtained in practice is usually time-series data collected by soft sensors, which are highly nonlinear, nonstationary, imbalanced, and noisy. Most existing soft-sensing machine learning models focus on capturing either intra-series temporal dependencies or pre-defined inter-series correlations, while ignoring the correlation between labels as each instance is associated with multiple labels simultaneously. In this paper,…
A Hybrid Multi-Factor Network with Dynamic Sequence Modeling for Early Warning of Intraoperative Hypotension
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China（认知智能国家重点实验室，中国科学技术大学）；Intelligent Interconnected Systems Laboratory of Anhui Province, Hefei University of Technology（安徽省智能互联系统实验室，合肥工业大学）
A Multi-Scale Decomposition MLP-Mixer for Time Series Analysis
The Hong Kong University of Science and Technology（香港科技大学）；BNU-HKBU United International College（北京师范大学-香港浸会大学联合国际学院）；Guangzhou Urban Planning and Design Survey Research Institute（广州市城市规划勘测设计研究院）
Toward Foundation Model for Multivariate Wearable Sensing of Physiological Signals
Halıcıoğlu Data Science Institute, University of California San Diego（哈利奇奥格鲁数据科学研究所，加州大学圣地亚哥分校）；Schools of Engineering, Arizona State University（工程学院，亚利桑那州立大学）
Neurophysiological time series, such as electromyographic signal and intracortical recordings, are typically composed of many individual spiking sources, the recovery of which can give fundamental insights into the biological system of interest or provide neural information for man-machine interfaces. For this reason, source separation algorithms have become an increasingly important tool in neuroscience and neuroengineering. However, in noisy or highly multivariate recordings these decomposition techniques often make a large number of errors, which degrades human-machine interfacing applicati…
