Natural language is inherently a discrete symbolic representation of human knowledge. Recent advances in machine learning (ML) and in natural language processing (NLP) seem to contradict the above intuition: discrete symbols are fading away, erased by vectors or tensors called distributed and distributional representations. However, there is a strict link between distributed/distributional representations and discrete symbols, being the first an approximation of the second. A clearer understanding of the strict link between distributed/distributional representations and symbols may certainly lead to radically new deep learning networks. In this paper we make a survey that aims to renew the link between symbolic representations and distributed/distributional representations. This is the right time to revitalize the area of interpreting how discrete symbols are represented inside neural networks.
Bridging Compositional and Distributional Semantics: A Survey on Latent Semantic Geometry via AutoEncoder
Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；Idiap Research Institute（Idiap研究 institute）；Cancer Biomarker Centre, CRUK Manchester Institute（CRUK曼彻斯特癌症生物标志物中心）
Quasi-symbolic Semantic Geometry over Transformer-based Variational AutoEncoder
Department of Computer Science, University of Manchester（曼彻斯特大学计算机科学系）；Idiap Research Institute（Idiap研究机构）；National Biomarker Centre, CRUK-MI, Univ. of Manchester（国家生物标记中心，CRUK-MI，曼彻斯特大学）
NL2GenSym: Natural Language to Generative Symbolic Rules for SOAR Cognitive Architecture via Large Language Models
College of Systems Engineering, National University of Defense Technology（系统工程学院，国防科技大学）；State Key Laboratory of Digital Intelligent Modeling and Simulation（数字智能建模与仿真国家重点实验室）；Test Center, National University of Defense Technology（测试中心，国防科技大学）
Do Large Language Models Advocate for Inferentialism?
College of Media Arts, Science and Technology, School of Informatics, University of Tsukuba（筑波大学媒体艺术、科学与技术学院，信息学院）；Institute of Systems and Information Engineering, University of Tsukuba（筑波大学系统与信息工程研究所）
