A graph neural network transforms features in each vertex's neighborhood into a vector representation of the vertex. Afterward, each vertex's representation is used independently for predicting its label. This standard pipeline implicitly assumes that vertex labels are conditionally independent given their neighborhood features. However, this is a strong assumption, and we show that it is far from true on many real-world graph datasets. Focusing on regression tasks, we find that this conditional independence assumption severely limits predictive power. This should not be that surprising, given that traditional graph-based semi-supervised learning methods such as label propagation work in the opposite fashion by explicitly modeling the correlation in predicted outcomes.
Here, we address this problem with an interpretable and efficient framework that can improve any graph neural network architecture simply by exploiting correlation structure in the regression residuals. In particular, we model the joint distribution of residuals on vertices with a parameterized multivariate Gaussian, and estimate the parameters by maximizing the marginal likelihood of the observed labels. Our framework achieves substantially higher accuracy than competing baselines, and the learned parameters can be interpreted as the strength of correlation among connected vertices. Furthermore, we develop linear time algorithms for low-variance, unbiased model parameter estimates, allowing us to scale to large networks. We also provide a basic version of our method that makes stronger assumptions on correlation structure but is painless to implement, often leading to great practical performance with minimal overhead.
LAIR-Net: Leaky Alignment-Impulse Residual Networks for Tabular Regression
Indian Institute of Technology Guwahati(印度技术学院加瓦哈蒂分校);SAFIR, Sorbonne University Abu Dhabi(阿布扎比索邦大学 SAFIR);Indian Institute of Technology Bombay(印度技术学院孟买分校)
Dept. of Computer Science, Mathematics, Physics and Statistics, University of British Columbia(计算机科学、数学、物理与统计学系,不列颠哥伦比亚大学);Dept. of Statistical Sciences, University of Toronto(统计学系,多伦多大学)
针对非高斯噪声下局部最小二乘估计失效的问题,提出局部多项式 Lp-范数回归方法,通过加权 Lp-范数估计进行局部拟合,并引入从残差估计参数 p 的新方法,在一维及二维数据中均表现优越。