Spectral-based graph neural networks (SGNNs) have been attracting increasing attention in graph representation learning. However, existing SGNNs are limited in implementing graph filters with rigid transforms (e.g., graph Fourier or predefined graph wavelet transforms) and cannot adapt to signals residing on graphs and tasks at hand. In this paper, we propose a novel class of graph neural networks that realizes graph filters with adaptive graph wavelets. Specifically, the adaptive graph wavelets are learned with neural network-parameterized lifting structures, where structure-aware attention-based lifting operations (i.e., prediction and update operations) are developed to jointly consider graph structures and node features. We propose to lift based on diffusion wavelets to alleviate the structural information loss induced by partitioning non-bipartite graphs. By design, the locality and sparsity of the resulting wavelet transform as well as the scalability of the lifting structure are guaranteed. We further derive a soft-thresholding filtering operation by learning sparse graph representations in terms of the learned wavelets, yielding a localized, efficient, and scalable wavelet-based graph filters. To ensure that the learned graph representations are invariant to node permutations, a layer is employed at the input of the networks to reorder the nodes according to their local topology information. We evaluate the proposed networks in both node-level and graph-level representation learning tasks on benchmark citation and bioinformatics graph datasets. Extensive experiments demonstrate the superiority of the proposed networks over existing SGNNs in terms of accuracy, efficiency, and scalability.
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);University of Chinese Academy of Sciences(中国科学院大学);Renmin University of China(中国人民大学)
Adaptive wavelet distillation from neural networks through interpretations
UC Berkeley(加州大学伯克利分校);CEA(法国原子能委员会);CNRS(法国国家科学研究中心);Université Paris-Saclay(巴黎-萨克雷大学);Université Paris Diderot(巴黎狄德罗大学);Sorbonne Paris Cité(索邦巴黎西岱大学)
Universidad Nacional de Colombia(哥伦比亚国立大学);The University of Tokyo(东京大学);McGill University(麦吉尔大学);Target Corporation(塔吉特公司);Digital Frontier Inc.(数字前沿公司)
Beyond Fixed Patches: Enhancing GPTs for Financial Prediction with Adaptive Segmentation and Learnable Wavelets
School of Computer Science and Technology, Tongji University(同济大学计算机科学与技术学院);School of Mathematical Sciences, Tongji University(同济大学数学科学学院);Seek Data Group, Emoney Inc.(Seek Data集团,Emoney公司)
AWEMixer: Adaptive Wavelet-Enhanced Mixer Network for Long-Term Time Series Forecasting
School of Computer Science and Technology, Xi’an Jiaotong University(西安交通大学计算机科学与技术学院);Department of Research and Development, Shandong New Beiyang Information Technology Co., Ltd.(山东新北洋信息技术有限公司);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术学院)
Adaptive ship-radiated noise recognition with learnable fine-grained wavelet transform
Key Laboratory of Speech Acoustics and Content Understanding, Institute of Acoustics, Chinese Academy of Sciences(中国科学院声学研究所);University of Chinese Academy of Sciences(中国科学院大学)
LiNO: Lifting based multiresolution neural operator
Department of Computational and Data Sciences, Indian Institute of Science(计算与数据科学系,印度科学研究所);IISc Mathematics Initiative, Department of Mathematics, Indian Institute of Science(印度科学研究所数学倡议,数学系)