The present paper reviews and discusses work from computer science that proposes to identify concepts in internal representations (hidden layers) of DNNs. It is examined, first, how existing methods actually identify concepts that are supposedly represented in DNNs. Second, it is discussed how conceptual spaces -- sets of concepts in internal representations -- are shaped by a tradeoff between predictive accuracy and compression. These issues are critically examined by drawing on philosophy. While there is evidence that DNNs able to represent non-trivial inferential relations between concepts, our ability to identify concepts is severely limited.
Random weights of DNNs and emergence of fixed points
Department of Mathematics, Pennsylvania State University, University Park, Pennsylvania, 16802, USA(宾夕法尼亚州立大学数学系);B. Verkin Institute for Low Temperature Physics(B. Verkin低温物理工程研究所);Engineering of the National Academy of Sciences of Ukraine, Nauky Ave., 47, Kharkiv, 61103, Ukraine(乌克兰国家科学院B. Verkin低温物理工程研究所)
On Background Bias of Post-Hoc Concept Embeddings in Computer Vision DNNs
University of Lübeck(吕贝克大学);Hochschule Anhalt(安哈尔特应用技术大学);Continental AG(大陆集团);University of Liverpool(利物浦大学);University of Wuppertal(伍珀塔尔大学);Technical University of Munich(慕尼黑工业大学)
Guiding Sparse Neural Networks with Neurobiological Principles to Elicit Biologically Plausible Representations
KEIM Institute, Albstadt-Sigmaringen University, Germany(KEIM研究所,阿尔布施塔特-西格马林根大学,德国);Department of Computer Science, Chemnitz University of Technology, Germany(计算机科学系, Chemnitz技术大学,德国)
A Concept-based approach to Voice Disorder Detection
Politecnico di Torino;CENTAI Institute(CENTAI研究院);San Feliciano Hospital(San Feliciano医院);SCDU Otorinolaringoiatria, Head Neck Cancer Unit, Ospedale San Giovanni Bosco(SCDU耳鼻喉科,头颈癌症单元,San Giovanni Bosco医院);Dipartimento di Oncologia, Università degli Studi di Torino(肿瘤学系,托里尼大学)
From Colors to Classes: Emergence of Concepts in Vision Transformers
Technical University of Denmark(丹麦技术大学);UiT The Arctic University of Norway(挪威北极大学);University of Copenhagen(哥本哈根大学);Norwegian Computing Center(挪威计算中心)
Human-like Cognitive Generalization for Large Models via Brain-in-the-loop Supervision
College of Computer Science and Technology, Zhejiang University(浙江大学计算机科学与技术学院);State Key Lab of Brain-Machine Intelligence, Zhejiang University(浙江大学脑机智能全国重点实验室)
An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications
Universidad de Cuenca(昆卡大学);Bauhaus-Universität Weimar(魏玛包豪斯大学);Leibniz Universität Hannover(汉诺威莱布尼茨大学);School of Engineering and Departamento de Recursos Hídricos y Ciencias Ambientales, Universidad de Cuenca(昆卡大学工程学院和水资源与环境科学系)
The Emergence of Organizing Structure in Conceptual Representation
Center for Data Science, New York University(纽约大学数据科学中心);Department of Psychology, New York University(纽约大学心理学系);Department of Computer Science, University of Sheffield(谢菲尔德大学计算机科学系);Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology(麻省理工学院脑与认知科学系);Center for Brains Minds and Machines(大脑心智与机器中心)
Debugging and Runtime Analysis of Neural Networks with VLMs (A Case Study)
University of Toronto(多伦多大学);KBR Inc.(KBR公司);NASA Ames(艾姆斯研究中心);Carnegie Mellon Univ.(卡内基梅隆大学);VMware Research(威睿研究院);Colorado State University(科罗拉多州立大学);SRI International(SRI国际公司)