ReSi: A Comprehensive Benchmark for Representational Similarity Measures
University of Passau（帕绍大学）；German Cancer Research Center (DKFZ)（德国癌症研究中心）；Helmholtz Imaging（亥姆霍兹成像中心）；University of Heidelberg（海德堡大学）；University of Mannheim（曼海姆大学）；RWTH Aachen University（亚琛工业大学）；Heidelberg University Hospital（海德堡大学医院）；National Center for Tumor Diseases (NCT) Heidelberg（海德堡国家肿瘤疾病中心）；GESIS - Leibniz Institute for the Social Sciences（莱布尼茨社会科学研究所GESIS）；Complexity Science Hub（复杂性科学中心）
Max Klabunde, Tassilo Wald, Tobias Schumacher, Klaus Maier-Hein, Markus Strohmaier, Florian Lemmerich
Measuring the similarity of different representations of neural architectures is a fundamental task and an open research challenge for the machine learning community. This paper presents the first comprehensive benchmark for evaluating representational similarity measures based on well-defined groundings of similarity. The representational similarity (ReSi) benchmark consists of (i) six carefully designed tests for similarity measures, (ii) 24 similarity measures, (iii) 14 neural network architectures, and (iv) seven datasets, spanning over the graph, language, and vision domains. The benchmark opens up several important avenues of research on representational similarity that enable novel explorations and applications of neural architectures. We demonstrate the utility of the ReSi benchmark by conducting experiments on various neural network architectures, real world datasets and similarity measures. All components of the benchmark are publicly available and thereby facilitate systematic reproduction and production of research results. The benchmark is extensible, future research can build on and further expand it. We believe that the ReSi benchmark can serve as a sound platform catalyzing future research that aims to systematically evaluate existing and explore novel ways of comparing representations of neural architectures.
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）；University of Alberta（阿尔伯塔大学）；Purdue University Fort Wayne（普渡大学韦恩堡分校）
Similarity analysis is one of the crucial steps in most fMRI studies. Representational Similarity Analysis (RSA) can measure similarities of neural signatures generated by different cognitive states. This paper develops Deep Representational Similarity Learning (DRSL), a deep extension of RSA that is appropriate for analyzing similarities between various cognitive tasks in fMRI datasets with a large number of subjects, and high-dimensionality -- such as whole-brain images. Unlike the previous methods, DRSL is not limited by a linear transformation or a restricted fixed nonlinear kernel functio…
Nanjing University of Aeronautics and Astronautics（南京航空航天大学）
Representational Similarity Analysis (RSA) aims to explore similarities between neural activities of different stimuli. Classical RSA techniques employ the inverse of the covariance matrix to explore a linear model between the neural activities and task events. However, calculating the inverse of a large-scale covariance matrix is time-consuming and can reduce the stability and robustness of the final analysis. Notably, it becomes severe when the number of samples is too large. For facing this shortcoming, this paper proposes a novel RSA method called gradient-based RSA (GRSA). Moreover, the p…
Similarity of Neural Network Models: A Survey of Functional and Representational Measures
University of Passau（帕绍大学）；University of Mannheim（曼海姆大学）；RWTH Aachen University（亚琛工业大学）；GESIS - Leibniz Institute for the Social Sciences（GESIS莱布尼茨社会科学研究所）；Complexity Science Hub Vienna（维也纳复杂性科学中心）
School of Computing（计算机学院）；University of Kent（肯特大学）
The ability of Variational Autoencoders (VAEs) to learn disentangled representations has made them popular for practical applications. However, their behaviour is not yet fully understood. For example, the questions of when they can provide disentangled representations, or suffer from posterior collapse are still areas of active research. Despite this, there are no layerwise comparisons of the representations learned by VAEs, which would further our understanding of these models. In this paper, we thus look into the internal behaviour of VAEs using representational similarity techniques. Speci…
Similarity measures are a vital tool for understanding how language models represent and process language. Standard representational similarity measures such as cosine similarity and Euclidean distance have been successfully used in static word embedding models to understand how words cluster in semantic space. Recently, these measures have been applied to embeddings from contextualized models such as BERT and GPT-2. In this work, we call into question the informativity of such measures for contextualized language models. We find that a small number of rogue dimensions, often just 1-3, dominat…
When Does Closeness in Distribution Imply Representational Similarity? An Identifiability Perspective
Technical University of Denmark（丹麦技术大学）；University of Trento, Italy（意大利特伦托大学）；Helmholtz AI, Munich（慕尼黑海德堡人工智能研究所）；Technical University of Munich（慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；University of Copenhagen（哥本哈根大学）
Neural networks trained on standard image classification data sets are shown to be less resistant to data set bias. It is necessary to comprehend the behavior objective function that might correspond to superior performance for data with biases. However, there is little research on the selection of the objective function and its representational structure when trained on data set with biases. In this paper, we investigate the performance and internal representational structure of convolution-based neural networks (e.g., ResNets) trained on biased data using various objective functions. We spec…
Geometry matters: insights from Ollivier Ricci Curvature and Ricci Flow into representational alignment through Ollivier-Ricci Curvature and Ricci Flow
Max Planck Institute for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所）；Berlin School of Mind and Brain（柏林心智与脑学校）；Humboldt-Universität zu Berlin（洪堡大学）；Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI)（可扩展数据与人工智能中心（ScaDS.AI））
Measuring the Measures: Discriminative Capacity of Representational Similarity Metrics Across Model Families
Department of Computer Science and Engineering, UC San Diego（计算机科学与工程系，UC San Diego）；Department of Electrical and Computer Engineering, UC San Diego（电气与计算机工程系，UC San Diego）；Department of Cognitive Science, UC San Diego（认知科学系，UC San Diego）
Barycentric alignment for instance-level comparison of neural representations
Department of XXX, University of YYY, Location, Country（YYY大学XXX系）；School of ZZZ, Institute of WWW, Location, Country（WWW研究所ZZZ学院）；Department of Electrical and Computer Engineering, UCSD（UCSD电子与计算机工程系）；Cognitive Science Department, UCSD（UCSD认知科学系）；Department of Computer Science（计算机科学系）
Flatiron Institute（熨斗研究所）；New York University（纽约大学）
Quantifying similarity between neural representations -- e.g. hidden layer activation vectors -- is a perennial problem in deep learning and neuroscience research. Existing methods compare deterministic responses (e.g. artificial networks that lack stochastic layers) or averaged responses (e.g., trial-averaged firing rates in biological data). However, these measures of _deterministic_ representational similarity ignore the scale and geometric structure of noise, both of which play important roles in neural computation. To rectify this, we generalize previously proposed shape metrics (Williams…
DeepMind（深度思维）；Facebook AI Research (FAIR)（脸书人工智能研究院）；Google Brain（谷歌大脑）；Cornell University（康奈尔大学）
Comparing different neural network representations and determining how representations evolve over time remain challenging open questions in our understanding of the function of neural networks. Comparing representations in neural networks is fundamentally difficult as the structure of representations varies greatly, even across groups of networks trained on identical tasks, and over the course of training. Here, we develop projection weighted CCA (Canonical Correlation Analysis) as a tool for understanding neural networks, building off of SVCCA, a recently proposed method (Raghu et al., 2017)…
Different neural networks trained on the same dataset often learn similar input-output mappings with very different weights. Is there some correspondence between these neural network solutions? For linear networks, it has been shown that different instances of the same network architecture encode the same representational similarity matrix, and their neural activity patterns are connected by orthogonal transformations. However, it is unclear if this holds for non-linear networks. Using a shared response model, we show that different neural networks encode the same input examples as different o…
Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA 10027（美国纽约州纽约市哥伦比亚大学祖克曼心智脑行为研究所）
Representational similarity analysis (RSA) has been shown to be an effective framework to characterize brain-activity profiles and deep neural network activations as representational geometry by computing the pairwise distances of the response patterns as a representational dissimilarity matrix (RDM). However, how to properly analyze and visualize the representational geometry as dynamics over the time course from stimulus onset to offset is not well understood. In this work, we formulated the pipeline to understand representational dynamics with RDM movies and Procrustes-aligned Multidimensio…
Aalto University（阿尔托大学）；University of Manchester（曼彻斯特大学）
Similarity metrics such as representational similarity analysis (RSA) and centered kernel alignment (CKA) have been used to compare layer-wise representations between neural networks. However, these metrics are confounded by the population structure of data items in the input space, leading to spuriously high similarity for even completely random neural networks and inconsistent domain relations in transfer learning. We introduce a simple and generally applicable fix to adjust for the confounder with covariate adjustment regression, which retains the intuitive invariance properties of the orig…
Recent work has sought to understand the behavior of neural networks by comparing representations between layers and between different trained models. We examine methods for comparing neural network representations based on canonical correlation analysis (CCA). We show that CCA belongs to a family of statistics for measuring multivariate similarity, but that neither CCA nor any other statistic that is invariant to invertible linear transformation can measure meaningful similarities between representations of higher dimension than the number of data points. We introduce a similarity index that …
University of Maryland, College Park（马里兰大学帕克分校）；Max Planck Institute for Software Systems（马克斯·普朗克软件系统研究所）；PUCRS（巴西圣保罗天主教大学）；The Alan Turing Institute（艾伦·图灵研究所）；University of Cambridge（剑桥大学）
A major challenge in studying robustness in deep learning is defining the set of ``meaningless'' perturbations to which a given Neural Network (NN) should be invariant. Most work on robustness implicitly uses a human as the reference model to define such perturbations. Our work offers a new view on robustness by using another reference NN to define the set of perturbations a given NN should be invariant to, thus generalizing the reliance on a reference ``human NN'' to any NN. This makes measuring robustness equivalent to measuring the extent to which two NNs share invariances, for which we pro…
BRAMS；CRBLM；Mila（米拉研究所）；Université de Montréal（蒙特利尔大学）；University of Leipzig（莱比锡大学）
Centered Kernel Alignment (CKA) was recently proposed as a similarity metric for comparing activation patterns in deep networks. Here we experiment with the modified RV-coefficient (RV2), which has very similar properties as CKA while being less sensitive to dataset size. We compare the representations of networks that received varying amounts of training on different layers: a standard trained network (all parameters updated at every step), a freeze trained network (layers gradually frozen during training), random networks (only some layers trained), and a completely untrained network. We fou…
Partial Soft-Matching Distance for Neural Representational Comparison with Partial Unit Correspondence
Department of Cognitive Science University of California, San Diego（认知科学系加州大学圣地亚哥分校）；Center for Neural Science, New York University（纽约大学神经科学中心）；Center for Computational Neuroscience, Flatiron Institute（Flatiron研究所计算神经科学中心）；Department of Cognitive Science Department of Computer Science and Engineering University of California, San Diego（认知科学系计算机科学与工程系加州大学圣地亚哥分校）
To address the limitations of existing magnitude-based pruning algorithms in cases where model weights or activations are of large and similar magnitude, we propose a novel perspective to discover parameter redundancy among channels and accelerate deep CNNs via channel pruning. Precisely, we argue that channels revealing similar feature information have functional overlap and that most channels within each such similarity group can be removed without compromising model's representational power. After deriving an effective metric for evaluating channel similarity through probabilistic modeling,…
University of Pennsylvania（宾夕法尼亚大学）；Mila Québec AI Institute（米拉魁北克人工智能研究所）；McGill University（麦吉尔大学）
Deep neural networks implement a sequence of layer-by-layer operations that are each relatively easy to understand, but the resulting overall computation is generally difficult to understand. We consider a simple hypothesis for interpreting the layer-by-layer construction of useful representations: perhaps the role of each layer is to reformat information to reduce the "distance" to the desired outputs. With this framework, the layer-wise computation implemented by a deep neural network can be viewed as a path through a high-dimensional representation space. We formalize this intuitive idea of…
Achieving More Human Brain-Like Vision via Human EEG Representational Alignment
Department of Psychology, The Ohio State University（心理学系，俄亥俄州立大学）；McGovern Institute for Brain Research, Massachusetts Institute of Technology（脑科学研究所，麻省理工学院）；Department of Neuroscience, The University of Texas at Dallas（神经科学系，德克萨斯大学达拉斯分校）
University of Tübingen（蒂宾根大学）；Baylor College of Medicine（贝勒医学院）；Rice University（莱斯大学）
Despite impressive performance on numerous visual tasks, Convolutional Neural Networks (CNNs) --- unlike brains --- are often highly sensitive to small perturbations of their input, e.g. adversarial noise leading to erroneous decisions. We propose to regularize CNNs using large-scale neuroscience data to learn more robust neural features in terms of representational similarity. We presented natural images to mice and measured the responses of thousands of neurons from cortical visual areas. Next, we denoised the notoriously variable neural activity using strong predictive models trained on thi…
Princeton University（普林斯顿大学）；NYU Center for Data Science（纽约大学数据科学中心）；TU Berlin（柏林工业大学）；BIFOLD（柏林人工智能与机器学习数据科学基础设施中心）；Google DeepMind（谷歌DeepMind）；MPI for Human Cognitive and Brain Sciences（马克斯·普朗克人类认知与脑科学研究所）；University of Cambridge（剑桥大学）；MIT（麻省理工学院）；UCL（伦敦大学学院）；University of Amsterdam（阿姆斯特丹大学）；Intel Labs（英特尔实验室）；Columbia University（哥伦比亚大学）；MPI for Empirical Aesthetics（马克斯·普朗克经验美学研究所）；Google Research（谷歌研究院）；Anthropic（Anthropic公司）；Harvard University（哈佛大学）
Soft Matching Distance: A metric on neural representations that captures single-neuron tuning
McGovern Institute for Brain Research（麦戈文脑研究所）；Massachusetts Institute for Technology（麻省理工学院）；Center for Neural Science, New York University（纽约大学神经科学中心）；Center for Computational Neuroscience（计算神经科学中心）；Flatiron Institute（熨斗研究所）
Decoupling Semantic Similarity from Spatial Alignment for Neural Networks
German Cancer Research Center (DKFZ)（德国癌症研究中心）；Helmholtz Imaging（亥姆霍兹成像）；University of Heidelberg（海德堡大学）；National Center for Tumor Diseases (NCT) Heidelberg（海德堡国家肿瘤疾病中心）
