As breakthroughs in deep learning transform key industries, models are increasingly required to extrapolate on datapoints found outside the range of the training set, a challenge we coin as out-of-support (OoS) generalisation. However, neural networks frequently exhibit catastrophic failure on OoS samples, yielding unrealistic but overconfident predictions. We address this challenge by reformulating the OoS generalisation problem as a sequence modelling task in the weight space, wherein the training set is partitioned into concentric shells corresponding to discrete sequential steps. Our WeightCaster framework yields plausible, interpretable, and uncertainty-aware predictions without necessitating explicit inductive biases, all the while maintaining high computational efficiency. Emprical validation on a synthetic cosine dataset and real-world air quality sensor readings demonstrates performance competitive or superior to the state-of-the-art. By enhancing reliability beyond in-distribution scenarios, these results hold significant implications for the wider adoption of artificial intelligence in safety-critical applications.
How does Weight Correlation Affect the Generalisation Ability of Deep Neural Networks
University of Liverpool(利物浦大学);State Key Laboratory of Computer Science, Institute of Software, CAS(中国科学院软件研究所计算机科学国家重点实验室);University of Chinese Academy of Sciences(中国科学院大学);Institute of Intellegence Software, Guangzhou(广州智能软件研究所)
Physics-informed neural operator for parametric phase-field modelling of interfacial degradation and microstructural evolution
Tongji University, College of Civil Engineering(同济大学,土木工程学院);Tongji University, State Key Laboratory of Disaster Reduction in Civil Engineering(同济大学,灾害防治工程重点实验室)
Index Set Fourier Series Features for Approximating Multi-dimensional Periodic Kernels
School of Information Technologies, The University of Sydney(悉尼大学信息技术学院);The University of Sydney(悉尼大学)
周期性经常在时间序列建模中使用自回归方法进行研究,但在内核文献中不太受欢迎,特别是对于更高维的问题,如纹理,晶体学和量子力学。大型数据集通常使得建模周期性对于其他强大的非参数方法(如高斯过程(GP))来说是站不住脚的,高斯过程(GP)通常会产生$\mathcal{O}(N^3)$计算负担,因此无法扩展到更大的数据集。为此,我们引入了一种称为\emph{Index Set Fourier Series Features}的方法,可追溯地利用多元傅立叶级数,并有效地将高维数据的周期核分解成一系列的基函数。我们表明,我们的近似产生显着更少的预测误差比替代方法,如基于随机傅立叶特征,并实现更好的概括与周期性数据的回归问题。