arXivDaily arXiv每日学术速递 周一至周五更新
北美退伍军人节(Veterans Day),12 日无更新
arXiv 2209.08418cs.LG
收藏

基于单个确定性神经网络的样本化不确定性量化

Sample-based Uncertainty Quantification with a Single Deterministic Neural Network

Hitachi America, Ltd.(日立美国有限公司)

Takuya Kanazawa, Chetan Gupta

AI总结:

本文提出一种改进的 DISCO Nets 神经架构,以紧凑低维噪声向量实现快速稳定的训练,并在表格数据回归上验证其在不确定性量化和点预测方面优于或媲美现有 UQ 基线。

摘要:

开发一种准确、灵活且数值高效的不确定性量化(UQ)方法是机器学习中的基础性挑战之一。此前,一种名为 DISCO Nets 的 UQ 方法已被提出(Bouchacourt et al., 2016),该方法通过最小化能量分数来训练神经网络。在该方法中,一个位于 $\mathbb{R}^{10\text{--}100}$ 的随机噪声向量与原始输入向量拼接,从而在仅使用单个神经网络的情况下产生多样化的集合预测。尽管该方法在计算机视觉中的手部姿态估计任务上表现出有前景的性能,但它是否能在表格数据的回归任务上同样良好地工作,以及它如何与 NGBoost 等更近期的高级 UQ 方法竞争,仍未得到探索。在本文中,我们提出了一种改进的 DISCO Nets 神经架构,该架构能够实现更快、更稳定的训练,同时仅使用维度约为 $\sim \mathcal{O}(1)$ 的紧凑噪声向量。我们在多种真实世界表格数据集上对该方法进行了基准测试,并确认其与标准 UQ 基线相比具有竞争力甚至更优。此外,我们观察到,与使用传统均方误差训练的相同规模神经网络相比,它表现出更好的点预测性能。作为所提方法的另一优势,我们表明诸如 SHAP 之类的局部特征重要性计算方法可以容易地应用于预测分布的任意子区域。我们还提供了一个新的初等证明,说明使用能量分数学习预测分布的有效性。

英文摘要:

Development of an accurate, flexible, and numerically efficient uncertainty quantification (UQ) method is one of fundamental challenges in machine learning. Previously, a UQ method called DISCO Nets has been proposed (Bouchacourt et al., 2016), which trains a neural network by minimizing the energy score. In this method, a random noise vector in $\mathbb{R}^{10\text{--}100}$ is concatenated with the original input vector in order to produce a diverse ensemble forecast despite using a single neural network. While this method has shown promising performance on a hand pose estimation task in computer vision, it remained unexplored whether this method works as nicely for regression on tabular data, and how it competes with more recent advanced UQ methods such as NGBoost. In this paper, we propose an improved neural architecture of DISCO Nets that admits faster and more stable training while only using a compact noise vector of dimension $\sim \mathcal{O}(1)$. We benchmark this approach on miscellaneous real-world tabular datasets and confirm that it is competitive with or even superior to standard UQ baselines. Moreover we observe that it exhibits better point forecast performance than a neural network of the same size trained with the conventional mean squared error. As another advantage of the proposed method, we show that local feature importance computation methods such as SHAP can be easily applied to any subregion of the predictive distribution. A new elementary proof for the validity of using the energy score to learn predictive distributions is also provided.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑