arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2108.01661cs.LGstat.ML
收藏

基于统计检验的接地表示相似性

Grounding Representation Similarity with Statistical Testing

University of California Berkeley(加州大学伯克利分校)

Frances Ding, Jean-Stanislas Denain, Jacob Steinhardt

摘要:

为了理解神经网络的行为,最近的工作定量地比较不同的网络的学习表示使用典型相关分析(CCA),中心核对齐(CKA),和其他相异的措施。不幸的是,这些被广泛使用的度量往往在基本观测上存在分歧,例如深度网络是否只在随机初始化方面有所不同,从而学习到类似的表示。这些分歧提出了一个问题:如果有的话,我们应该相信这些不同的措施?我们提供了一个框架,通过一个具体的测试接地这个问题:措施应该有敏感性的变化,影响功能行为,和特异性的变化,不。我们量化这通过各种功能的行为,包括探测精度和鲁棒性分布偏移,并检查变化,如不同的随机初始化和删除主成分。我们发现,目前的指标表现出不同的弱点,请注意,一个经典的基线表现令人惊讶的好,并突出设置,所有的指标似乎失败,从而提供了一个挑战集进一步改进。

英文摘要:

To understand neural network behavior, recent works quantitatively compare different networks' learned representations using canonical correlation analysis (CCA), centered kernel alignment (CKA), and other dissimilarity measures. Unfortunately, these widely used measures often disagree on fundamental observations, such as whether deep networks differing only in random initialization learn similar representations. These disagreements raise the question: which, if any, of these dissimilarity measures should we believe? We provide a framework to ground this question through a concrete test: measures should have sensitivity to changes that affect functional behavior, and specificity against changes that do not. We quantify this through a variety of functional behaviors including probing accuracy and robustness to distribution shift, and examine changes such as varying random initialization and deleting principal components. We find that current metrics exhibit different weaknesses, note that a classical baseline performs surprisingly well, and highlight settings where all metrics appear to fail, thus providing a challenge set for further improvement.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑