arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2310.00541stat.MLcs.LG
收藏

用于理解神经网络训练中变异性的鲁棒非参数假设检验

Robust Nonparametric Hypothesis Testing to Understand Variability in Training Neural Networks

Pacific Northwest National Laboratory(太平洋西北国家实验室)

Sinjini Banerjee, Reilly Cannon, Tim Marrinan, Tony Chiang, Anand D. Sarwate

AI总结:

针对深度神经网络训练因随机性产生的模型变异性,提出基于阈值前输出、依托鲁棒非参数假设检验框架的模型相近度衡量指标,可适配多种模型导出量。

摘要:

深度神经网络(DNN)的训练通常涉及随机优化,这意味着每次运行都会生成不同的模型。已有多项研究认为,当模型性能相同时(分类任务中即测试准确率一致),这种变异性可以忽略不计。然而,测试准确率相近的模型所计算的函数未必相同。我们基于网络阈值化前的输出,提出了一种衡量分类模型间相近程度的新指标。该指标基于鲁棒假设检验框架,还可适配于从训练后模型中导出的其他量化指标。

英文摘要:

Training a deep neural network (DNN) often involves stochastic optimization, which means each run will produce a different model. Several works suggest this variability is negligible when models have the same performance, which in the case of classification is test accuracy. However, models with similar test accuracy may not be computing the same function. We propose a new measure of closeness between classification models based on the output of the network before thresholding. Our measure is based on a robust hypothesis-testing framework and can be adapted to other quantities derived from trained models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑