Graph Neural Networks (GNNs) have proven to be highly effective for node classification tasks across diverse graph structural patterns. Traditionally, GNNs employ a uniform global filter, typically a low-pass filter for homophilic graphs and a high-pass filter for heterophilic graphs. However, real-world graphs often exhibit a complex mix of homophilic and heterophilic patterns, rendering a single global filter approach suboptimal. In this work, we theoretically demonstrate that a global filter optimized for one pattern can adversely affect performance on nodes with differing patterns. To address this, we introduce a novel GNN framework Node-MoE that utilizes a mixture of experts to adaptively select the appropriate filters for different nodes. Extensive experiments demonstrate the effectiveness of Node-MoE on both homophilic and heterophilic graphs.
Mixture of Weight-shared Heterogeneous Group Attention Experts for Dynamic Token-wise KV Optimization
University of Chinese Academy of Sciences（中国科学院大学）；Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences（中国科学院深圳先进技术研究院）；University of Macau（澳门大学）；Shenzhen University of Advanced Technology（深圳先进技术大学）
FlyLoRA: Boosting Task Decoupling and Parameter Efficiency via Implicit Rank-Wise Mixture-of-Experts
Department of Automation, Tsinghua University（自动化系，清华大学）；Academy of Medical Engineering and Translational Medicine, Tianjin University（医学工程与转化医学学院，天津大学）
Filtered not Mixed: Stochastic Filtering-Based Online Gating for Mixture of Large Language Models
University of Toronto（多伦多大学）；Vector Institute（矢量研究院）；Princeton University（普林斯顿大学）；McMaster University（麦克马斯特大学）；ETH Zürich（苏黎世联邦理工学院）；University of Oxford（牛津大学）；Dalhousie University（达尔豪斯大学）
This paper proposes a novel model for the rating prediction task in recommender systems which significantly outperforms previous state-of-the art models on a time-split Netflix data set. Our model is based on deep autoencoder with 6 layers and is trained end-to-end without any layer-wise pre-training. We empirically demonstrate that: a) deep autoencoder models generalize much better than the shallow ones, b) non-linear activation functions with negative parts are crucial for training deep models, and c) heavy use of regularization techniques such as dropout is necessary to prevent over-fiting.…
Tsinghua University（清华大学）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）
We propose a dynamic filtering strategy with large sampling field for ConvNets (LS-DFN), where the position-specific kernels learn from not only the identical position but also multiple sampled neighbor regions. During sampling, residual learning is introduced to ease training and an attention mechanism is applied to fuse features from different samples. Such multiple samples enlarge the kernels' receptive fields significantly without requiring more parameters. While LS-DFN inherits the advantages of DFN, namely avoiding feature map blurring by position-wise kernels while keeping translation i…
MP-MoE: Matrix Profile-Guided Mixture of Experts for Precipitation Forecasting
School of Electronics and Electrical Engineering, Hanoi University of Science and Technology（河内科技大学电子与电气工程学院）；Department of Electrical and Computer Engineering, University of Southern California, United States（美国南加州大学电气与计算机工程系）
Partition-wise Graph Filtering: A Unified Perspective Through the Lens of Graph Coarsening
Mohamed bin Zayed University of Artificial Intelligence（穆罕默德·本·扎耶德人工智能大学）；University of Chinese Academy of Sciences（中国科学院大学）；Hong Kong Baptist University（香港 Baptist 大学）
Graph Neural Networks (GNNs) exploit signals from node features and the input graph topology to improve node classification task performance. However, these models tend to perform poorly on heterophilic graphs, where connected nodes have different labels. Recently proposed GNNs work across graphs having varying levels of homophily. Among these, models relying on polynomial graph filters have shown promise. We observe that solutions to these polynomial graph filter models are also solutions to an overdetermined system of equations. It suggests that in some instances, the model needs to learn a …
Purdue University（普渡大学）；Indiana University（印第安纳大学）；Indiana University School of Medicine（印第安纳大学医学院）
Parameter estimation of mixture regression model using the expectation maximization (EM) algorithm is highly sensitive to outliers. Here we propose a fast and efficient robust mixture regression algorithm, called Component-wise Adaptive Trimming (CAT) method. We consider simultaneous outlier detection and robust parameter estimation to minimize the effect of outlier contamination. Robust mixture regression has many important applications including in human cancer genomics data, where the population often displays strong heterogeneity added by unwanted technological perturbations. Existing robu…
Iterative tensor network transformations for element-wise evaluation of elementary and filtering functions
Clarendon Laboratory, University of Oxford（牛津大学克拉伦登实验室）；The Hamburg Centre for Ultrafast Imaging（汉堡超快成像中心）；Institute for Quantum Physics, University of Hamburg（汉堡大学量子物理研究所）；Institute of Software Methods for Product Virtualization, German Aerospace Center (DLR)（德国航空航天中心产品虚拟化软件方法研究所）
Nanyang Technological University（南洋理工大学）；Delta-NTU Corporate Lab for Cyber-Physical Systems（Delta-南大网络物理系统联合实验室）；Delta Electronics Inc（台达电子公司）
In blind source separation of speech signals, the inherent imbalance in the source spectrum poses a challenge for methods that rely on single-source dominance for the estimation of the mixing matrix. We propose an algorithm based on the directional sparse filtering (DSF) framework that utilizes the Lehmer mean with learnable weights to adaptively account for source imbalance. Performance evaluation in multiple real acoustic environments show improvements in source separation compared to the baseline methods.
E-Learning systems (ELS) and Intelligent Tutoring Systems (ITS) play a significant part in today's education programs. Sequencing questions is the art of generating a personalized quiz for a target learner. A personalized test will enrich the learner's experience and will contribute to a more effective and efficient learning process. In this paper, we used the Neural Collaborative Filtering (NCF) model to generate question sequencing and compare it to a pair-wise memory-based question sequencing algorithm - EduRank. The NCF model showed significantly better ranking results than the EduRank mod…
The goal of this work is to investigate what singing voice separation approaches based on neural networks learn from the data. We examine the mapping functions of neural networks based on the denoising autoencoder (DAE) model that are conditioned on the mixture magnitude spectra. To approximate the mapping functions, we propose an algorithm inspired by the knowledge distillation, denoted the neural couplings algorithm (NCA). The NCA yields a matrix that expresses the mapping of the mixture to the target source magnitude information. Using the NCA, we examine the mapping functions of three fund…
It was recently shown that neural ordinary differential equation models cannot solve fundamental and seemingly straightforward tasks even with high-capacity vector field representations. This paper introduces two other fundamental tasks to the set that baseline methods cannot solve, and proposes mixtures of stochastic vector fields as a model class that is capable of solving these essential problems. Dynamic vector field selection is of critical importance for our model, and our approach is to propagate component uncertainty over the integration interval with a technique based on forward filte…
We introduce a novel and generic convolutional unit, DiCE unit, that is built using dimension-wise convolutions and dimension-wise fusion. The dimension-wise convolutions apply light-weight convolutional filtering across each dimension of the input tensor while dimension-wise fusion efficiently combines these dimension-wise representations; allowing the DiCE unit to efficiently encode spatial and channel-wise information contained in the input tensor. The DiCE unit is simple and can be seamlessly integrated with any architecture to improve its efficiency and performance. Compared to depth-wise…
We present a learning framework for reconstructing neural scene representations from a small number of unconstrained tourist photos. Since each image contains transient occluders, decomposing the static and transient components is necessary to construct radiance fields with such in-the-wild photographs where existing methods require a lot of training data. We introduce SF-NeRF, aiming to disentangle those two components with only a few images given, which exploits semantic information without any supervision. The proposed method contains an occlusion filtering module that predicts the transien…
Oden Institute for Computational Engineering & Sciences, the University of Texas at Austin（得克萨斯大学奥斯汀分校奥登计算工程与科学学院）；Dept of Aerospace Engineering & Engineering Mechanics, the University of Texas at Austin（得克萨斯大学奥斯汀分校航空航天工程与工程力学系）
Learning feature interactions is the key to success for the large-scale CTR prediction and recommendation. In practice, handcrafted feature engineering usually requires exhaustive searching. In order to reduce the high cost of human efforts in feature engineering, researchers propose several deep neural networks (DNN)-based approaches to learn the feature interactions in an end-to-end fashion. However, existing methods either do not learn both vector-wise interactions and bit-wise interactions simultaneously, or fail to combine them in a controllable manner. In this paper, we propose a new mod…
Learning Discriminators for Resampling in the Ensemble Gaussian Mixture Filter through a Normalizing Flow Approach
Department of Mathematics, The University of Hawaii at Manoa（夏威夷大学马诺阿分校数学系）；Department of Information and Computer Sciences, The University of Hawaii at Manoa（夏威夷大学马诺阿分校信息与计算机科学系）
National University of Singapore（新加坡国立大学）；Chengdu University of Information Technology（成都信息工程大学）；Northeast Petroleum University（东北石油大学）；Nanjing University of Science and Technology（南京理工大学）
In this work, we contribute a new multi-layer neural network architecture named ONCF to perform collaborative filtering. The idea is to use an outer product to explicitly model the pairwise correlations between the dimensions of the embedding space. In contrast to existing neural recommender models that combine user embedding and item embedding via a simple concatenation or element-wise product, our proposal of using outer product above the embedding layer results in a two-dimensional interaction map that is more expressive and semantically plausible. Above the interaction map obtained by oute…
Polynomial Neural Sheaf Diffusion: A Spectral Filtering Approach on Cellular Sheaves
Department of Computer Science and Technology, University of Cambridge（计算机科学与技术系，剑桥大学）；Department of Computer, Control and Management Engineering, Sapienza University（计算机、控制与管理工程系，萨皮恩扎大学）
