arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 1907.04223stat.MLcs.LG
收藏

描述深度学习模型的层间函数映射

Characterizing Inter-Layer Functional Mappings of Deep Learning Models

Air Force Research Laboratory(美国空军研究实验室);Space and Naval Warfare Systems Center Pacific(太平洋空间与海战系统中心);U.S. Army CCDC Aviation and Missile Center(美国陆军作战能力发展司令部航空与导弹中心);Dynetics, Inc.(戴奈蒂克斯公司);Modern Technology Solutions, Inc.(现代技术解决方案公司)

Donald Waagen, Katie Rainey, Jamie Gantert, David Gray, Megan King, M. Shane Thompson, Jonathan Barton, Will Waldron, Samantha Livingston, Don Hulsey

摘要:

深度学习架构已经证明了对象分类的最新性能,并且在商业产品中无处不在。这些方法通常在不理解(a)给定输入数据的分类任务的难度,以及(b)特定深度学习架构如何转换该数据的情况下应用。为了回答(a)和(b),我们举例说明了类分离的多变量非参数估计量Henze-Penrose(HP)统计量在原始表示和层诱导表示中的效用。给定一个$N$类的问题,我们的贡献定义的$C(N,2)$组合的HP统计作为一个样本,从分布的类对分离。这使我们能够描述在模型的每一层引起的类分离的分布变化。Fisher排列检验用于检测模型内的统计学显著变化。通过比较层之间的HP统计分布,可以统计地表征:训练期间的层适应,每个层对分类任务的贡献,以及训练和验证数据之间是否存在一致性。这是一个简单的深度神经网络,使用CIFAR 10与随机标签,CIFAR 10和MNIST数据集。

英文摘要:

Deep learning architectures have demonstrated state-of-the-art performance for object classification and have become ubiquitous in commercial products. These methods are often applied without understanding (a) the difficulty of a classification task given the input data, and (b) how a specific deep learning architecture transforms that data. To answer (a) and (b), we illustrate the utility of a multivariate nonparametric estimator of class separation, the Henze-Penrose (HP) statistic, in the original as well as layer-induced representations. Given an $N$-class problem, our contribution defines the $C(N,2)$ combinations of HP statistics as a sample from a distribution of class-pair separations. This allows us to characterize the distributional change to class separation induced at each layer of the model. Fisher permutation tests are used to detect statistically significant changes within a model. By comparing the HP statistic distributions between layers, one can statistically characterize: layer adaptation during training, the contribution of each layer to the classification task, and the presence or absence of consistency between training and validation data. This is demonstrated for a simple deep neural network using CIFAR10 with random-labels, CIFAR10, and MNIST datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑