流诱导对角高斯过程
Flow-Induced Diagonal Gaussian Processes
AI总结：
本研究提出FiD-GP压缩框架，通过归一化流先验、谱正则化及稳定投影机制实现权重不确定性的低维投影，可用于OoD检测，能大幅压缩模型、降低训练成本并保持SOTA性能。
摘要：
我们提出了Flow-Induced Diagonal Gaussian Processes（FiD-GP，流诱导对角高斯过程），这是一种压缩框架，它引入了紧凑的诱导权重矩阵，将神经网络的权重不确定性投影到低维子空间中。关键在于，FiD-GP依赖normalising-flow（归一化流）先验和谱正则化来增强其表达能力，并通过数值稳定的投影机制目标函数，使诱导子空间与特征梯度几何结构对齐。此外，我们展示了FiD-GP中的预测框架如何助力设计用于分布外（OoD）检测的单次投影。我们的分析表明，与基于SVGP（稀疏变分高斯过程）的基线相比，FiD-GP在各类任务上提升了不确定性估计能力，满足严格的谱残差界，具备理论保障的OoD检测性能，且能大幅压缩神经网络的存储需求，代价是推理计算量有所增加，增量取决于所使用的诱导权重数量。具体而言，在涵盖回归、图像分类、语义分割和分布外检测基准的全面实证研究中，它将贝叶斯训练成本降低了数个数量级，参数压缩约51%，模型体积减小约75%，同时达到了当前最优的准确率和不确定性估计水平。
英文摘要：
We present Flow-Induced Diagonal Gaussian Processes (FiD-GP), a compression framework that incorporates a compact inducing weight matrix to project a neural network's weight uncertainty into a lower-dimensional subspace. Critically, FiD-GP relies on normalising-flow priors and spectral regularisations to augment its expressiveness and align the inducing subspace with feature-gradient geometry through a numerically stable projection mechanism objective. Furthermore, we demonstrate how the prediction framework in FiD-GP can help to design a single-pass projection for Out-of-Distribution (OoD) detection. Our analysis shows that FiD-GP improves uncertainty estimation ability on various tasks compared with SVGP-based baselines, satisfies tight spectral residual bounds with theoretically guaranteed OoD detection, and significantly compresses the neural network's storage requirements at the cost of increased inference computation dependent on the number of inducing weights employed. Specifically, in a comprehensive empirical study spanning regression, image classification, semantic segmentation, and out-of-distribution detection benchmarks, it cuts Bayesian training cost by several orders of magnitude, compresses parameters by roughly 51%, reduces model size by about 75%, and matches state-of-the-art accuracy and uncertainty estimation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
