Despite recent studies on understanding deep neural networks (DNNs), there exists numerous questions on how DNNs generate their predictions. Especially, given similar predictions on different input samples, are the underlying mechanisms generating those predictions the same? In this work, we propose NeuCEPT, a method to locally discover critical neurons that play a major role in the model's predictions and identify model's mechanisms in generating those predictions. We first formulate a critical neurons identification problem as maximizing a sequence of mutual-information objectives and provide a theoretical framework to efficiently solve for critical neurons while keeping the precision under control. NeuCEPT next heuristically learns different model's mechanisms in an unsupervised manner. Our experimental results show that neurons identified by NeuCEPT not only have strong influence on the model's predictions but also hold meaningful information about model's mechanisms.
NeuronSeek: On Stability and Expressivity of Task-driven Neurons
Frontier of Artificial Networks (FAN) Lab, Department of Data Science, City University of Hong Kong(前沿人工智能网络实验室,数据科学系,香港城市大学);Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University(工业与系统工程系,香港理工大学);RIKEN Center for Advanced Intelligence Project(理化学研究所高级智能项目中心);Department of Applied Mathematics, The Hong Kong Polytechnic University(应用数学系,香港理工大学)
Sparse Interventions in Language Models with Differentiable Masking
University of Edinburgh(爱丁堡大学);Facebook AI Research(Facebook 人工智能研究院);University of Amsterdam(阿姆斯特丹大学);Innopolis University(因诺波利斯大学);University of Osnabrück(奥斯纳布吕克大学)
Learning Hybrid Biophysical Neuron Models with Neural ODEs
Hertie Institute for AI in Brain Health(人工智能与脑健康赫尔特研究所);Tübingen AI Center(图宾根人工智能中心);University of Tübingen(图宾根大学);Machine Learning in Science, Excellence Cluster Machine Learning, University of Tübingen(科学机器学习,机器学习卓越集群,图宾根大学);Max Planck Institute for Biological Intelligence, München, Germany(生物智能马克斯·普朗克研究所,慕尼黑,德国);Department Empirical Inference, Max Planck Institute for Intelligent Systems, Tübingen, Germany(经验推断部门,智能系统马克斯·普朗克研究所,图宾根,德国)
Fault-Tolerant Neural Networks from Biological Error Correction Codes
Center for Theoretical Physics, Massachusetts Institute of Technology(麻省理工学院理论物理中心);Department of Physics, Massachusetts Institute of Technology(麻省理工学院物理系);Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology(麻省理工学院电气工程与计算机科学系)