时空信号预测在众多领域中发挥着重要作用，尤其是在神经科学与交通运输领域。由于网络具有高度复杂的空间结构以及非线性的时序动态特性，该任务极具挑战性。为了对人类大脑和交通网络实现可靠且及时的预测，我们提出了快速时序小波图神经网络（FTWGNN），该方法在具有底层图结构的时序数据学习任务中兼具时间和内存效率，这得益于多分辨率分析理论以及离散空间上的小波理论。我们采用多分辨率矩阵分解（MMF）（Kondor et al., 2014）对高度稠密的图结构进行分解，并计算相应的稀疏小波基，从而能够构建快速小波卷积作为我们新型架构的主干。在真实世界的PEMS-BAY、METR-LA交通数据集和AJILE12 ECoG数据集上的实验结果表明，FTWGNN在保持较低计算开销的同时，与最先进的方法相比具有竞争力。我们的PyTorch实现已在 https://github.com/HySonLab/TWGNN 公开提供。
英文摘要：
Spatio-temporal signals forecasting plays an important role in numerous domains, especially in neuroscience and transportation. The task is challenging due to the highly intricate spatial structure, as well as the non-linear temporal dynamics of the network. To facilitate reliable and timely forecast for the human brain and traffic networks, we propose the Fast Temporal Wavelet Graph Neural Networks (FTWGNN) that is both time- and memory-efficient for learning tasks on timeseries data with the underlying graph structure, thanks to the theories of multiresolution analysis and wavelet theory on discrete spaces. We employ Multiresolution Matrix Factorization (MMF) (Kondor et al., 2014) to factorize the highly dense graph structure and compute the corresponding sparse wavelet basis that allows us to construct fast wavelet convolution as the backbone of our novel architecture. Experimental results on real-world PEMS-BAY, METR-LA traffic datasets and AJILE12 ECoG dataset show that FTWGNN is competitive with the state-of-the-arts while maintaining a low computational footprint. Our PyTorch implementation is publicly available at https://github.com/HySonLab/TWGNN
Spatiotemporal Forecasting of Traffic Flow using Wavelet-based Temporal Attention
Sorbonne University（索邦大学）；Indian Institute of Science, Education and Research, Kolkata（印度科学教育与研究学院（加尔各答））；International Institute of Information Technology, Bangalore（班加罗尔国际信息技术学院）
Localization of Dummy Data Injection Attacks in Power Systems Considering Incomplete Topological Information: A Spatio-Temporal Graph Wavelet Convolutional Neural Network Approach
Northeast Electric Power University（东北电力大学）；Jilin Electric Power Research Institute of State Grid（国家电网吉林电力科学研究院）；Northeast Electric Power Design and Institute Co., Ltd. of China Power Engineering Consulting Group（中国电力工程顾问集团东北电力设计院有限公司）；The Hong Kong University of Science and Technology（香港科技大学）
Hohai University（河海大学）；California State University, Fullerton（加州州立大学富勒顿分校）
Computational intelligence-based ocean characteristics forecasting applications, such as Significant Wave Height (SWH) prediction, are crucial for avoiding social and economic loss in coastal cities. Compared to the traditional empirical-based or numerical-based forecasting models, "soft computing" approaches, including machine learning and deep learning models, have shown numerous success in recent years. In this paper, we focus on enabling the deep learning model to learn both short-term and long-term spatial-temporal dependencies for SWH prediction. A Wavelet Graph Neural Network (WGNN) app…
Beijing University of Posts and Telecommunications（北京邮电大学）；Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；Institute for Interdisciplinary Information Sciences (IIIS), Tsinghua University（清华大学交叉信息研究院）
Traffic forecasting is crucial for public safety and resource optimization, yet is very challenging due to three aspects: i) current existing works mostly exploit intricate temporal patterns (e.g., the short-term thunderstorm and long-term daily trends) within a single method, which fail to accurately capture spatio-temporal dependencies under different schemas; ii) the under-exploration of the graph positional encoding limit the extraction of spatial information in the commonly used full graph attention network; iii) the quadratic complexity of the full graph attention introduces heavy comput…
York University（约克大学）；Borealis AI（北极光人工智能）；Ryerson University（瑞尔森大学）；Samsung AI Centre Toronto（三星人工智能中心多伦多分部）；Vector Institute（向量研究所）
Normalizing flows are a class of probabilistic generative models which allow for both fast density computation and efficient sampling and are effective at modelling complex distributions like images. A drawback among current methods is their significant training cost, sometimes requiring months of GPU training time to achieve state-of-the-art results. This paper introduces Wavelet Flow, a multi-scale, normalizing flow architecture based on wavelets. A Wavelet Flow has an explicit representation of signal scale that inherently includes models of lower resolution signals and conditional generati…
Beyond Fixed Patches: Enhancing GPTs for Financial Prediction with Adaptive Segmentation and Learnable Wavelets
School of Computer Science and Technology, Tongji University（同济大学计算机科学与技术学院）；School of Mathematical Sciences, Tongji University（同济大学数学科学学院）；Seek Data Group, Emoney Inc.（Seek Data集团，Emoney公司）
Northeast Normal University（东北师范大学）；University of Massachusetts（马萨诸塞大学）
The rapid spread of COVID-19 disease has had a significant impact on the world. In this paper, we study COVID-19 data interpretation and visualization using open-data sources for 351 cities and towns in Massachusetts from December 6, 2020 to September 25, 2021. Because cities are embedded in rather complex transportation networks, we construct the spatio-temporal dynamic graph model, in which the graph attention neural network is utilized as a deep learning method to learn the pandemic transition probability among major cities in Massachusetts. Using the spectral graph wavelet transform (SGWT)…
We present a logarithmic-scale efficient convolutional neural network architecture for edge devices, named WaveletNet. Our model is based on the well-known depthwise convolution, and on two new layers, which we introduce in this work: a wavelet convolution and a depthwise fast wavelet transform. By breaking the symmetry in channel dimensions and applying a fast algorithm, WaveletNet shrinks the complexity of convolutional blocks by an O(logD/D) factor, where D is the number of channels. Experiments on CIFAR-10 and ImageNet classification show superior and comparable performances of WaveletNet …
Multiresolution Matrix Factorization (MMF) is unusual amongst fast matrix factorization algorithms in that it does not make a low rank assumption. This makes MMF especially well suited to modeling certain types of graphs with complex multiscale or hierarchical strucutre. While MMF promises to yields a useful wavelet basis, finding the factorization itself is hard, and existing greedy methods tend to be brittle. In this paper we propose a learnable version of MMF that carfully optimizes the factorization with a combination of reinforcement learning and Stiefel manifold optimization through back…
AWEMixer: Adaptive Wavelet-Enhanced Mixer Network for Long-Term Time Series Forecasting
School of Computer Science and Technology, Xi’an Jiaotong University（西安交通大学计算机科学与技术学院）；Department of Research and Development, Shandong New Beiyang Information Technology Co., Ltd.（山东新北洋信息技术有限公司）；Department of Computer Science and Technology, Tsinghua University（清华大学计算机科学与技术学院）
This paper presents a end-to-end framework for the F0 transformation in the context of expressive voice conversion. A single neural network is proposed, in which a first module is used to learn F0 representation over different temporal scales and a second adversarial module is used to learn the transformation from one emotion to another. The first module is composed of a convolution layer with wavelet kernels so that the various temporal scales of F0 variations can be efficiently encoded. The single decomposition/transformation network allows to learn in a end-to-end manner the F0 decompositio…
Wavelet-Enhanced Neural ODE and Graph Attention for Interpretable Energy Forecasting
organization= School of Science, Engineering \& Technology, East Delta University , addressline = Abdullah Al Noman Road, Noman Society , city= Chattagram , postcode= 4209 , country= Bangladesh
University of Bonn（波恩大学）；Fraunhofer Center for Machine Learning and SCAI（弗劳恩霍夫机器学习与SCAI中心）；National University of Singapore（新加坡国立大学）
Wavelets are well known for data compression, yet have rarely been applied to the compression of neural networks. This paper shows how the fast wavelet transform can be used to compress linear layers in neural networks. Linear layers still occupy a significant portion of the parameters in recurrent neural networks (RNNs). Through our method, we can learn both the wavelet bases and corresponding coefficients to efficiently represent the linear layers of RNNs. Our wavelet compressed RNNs have significantly fewer parameters yet still perform competitively with the state-of-the-art on synthetic an…
Technical University of Munich（慕尼黑工业大学）；RWTH Aachen University（亚琛工业大学）
Generating highly detailed, complex data is a long-standing and frequently considered problem in the machine learning field. However, developing detail-aware generators remains an challenging and open problem. Generative adversarial networks are the basis of many state-of-the-art methods. However, they introduce a second network to be trained as a loss function, making the interpretation of the learned functions much more difficult. As an alternative, we present a new method based on a wavelet loss formulation, which remains transparent in terms of what is optimized. The wavelet-based loss fun…
Institute of Computing Technology, Chinese Academy of Sciences（中国科学院计算技术研究所）；University of Chinese Academy of Sciences（中国科学院大学）
We present graph wavelet neural network (GWNN), a novel graph convolutional neural network (CNN), leveraging graph wavelet transform to address the shortcomings of previous spectral graph CNN methods that depend on graph Fourier transform. Different from graph Fourier transform, graph wavelet transform can be obtained via a fast algorithm without requiring matrix eigendecomposition with high computational cost. Moreover, graph wavelets are sparse and localized in vertex domain, offering high efficiency and good interpretability for graph convolution. The proposed GWNN significantly outperforms…
