arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2602.13550cs.LG
收藏

通过权重空间序列建模实现超出支持泛化

Out-of-Support Generalisation via Weight-Space Sequence Modelling

University of Bristol(布里斯托大学)

Roussel Desmond Nzoyem

AI总结:

通过权重空间序列建模方法,提升模型对超出训练集范围数据的泛化能力与预测可靠性。

摘要:

随着深度学习的突破性进展改变关键行业,模型越来越多地需要对训练集范围之外的数据点进行外推,我们将其称为超出支持(OoS)泛化。然而,神经网络在OoS样本上经常出现灾难性失败,产生不现实但过于自信的预测。我们通过将OoS泛化问题重新表述为权重空间中的序列建模任务来解决这一挑战,其中训练集被划分为对应于离散序列步骤的同心壳层。我们的WeightCaster框架能够产生合理、可解释且具有不确定性意识的预测,而无需显式归纳偏置,同时保持高计算效率。在合成余弦数据集和现实世界的空气质量传感器读数上的实证验证表明,其性能与最先进的方法相当或更优。通过在分布内场景之外提升可靠性,这些结果对人工智能在安全关键应用中的广泛应用具有重要意义。

英文摘要:

As breakthroughs in deep learning transform key industries, models are increasingly required to extrapolate on datapoints found outside the range of the training set, a challenge we coin as out-of-support (OoS) generalisation. However, neural networks frequently exhibit catastrophic failure on OoS samples, yielding unrealistic but overconfident predictions. We address this challenge by reformulating the OoS generalisation problem as a sequence modelling task in the weight space, wherein the training set is partitioned into concentric shells corresponding to discrete sequential steps. Our WeightCaster framework yields plausible, interpretable, and uncertainty-aware predictions without necessitating explicit inductive biases, all the while maintaining high computational efficiency. Emprical validation on a synthetic cosine dataset and real-world air quality sensor readings demonstrates performance competitive or superior to the state-of-the-art. By enhancing reliability beyond in-distribution scenarios, these results hold significant implications for the wider adoption of artificial intelligence in safety-critical applications.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑