arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.05368cs.AIcs.LG
收藏

利用范畴论构建向量符号架构的基础

Developing a Foundation of Vector Symbolic Architectures Using Category Theory

Cheriton School of Computer Science University of Waterloo(滑铁卢大学切尔顿计算机科学学院);Centre for Theoretical Neuroscience Systems Design Engineering University of Waterloo(滑铁卢大学理论神经科学中心系统设计工程学院);Department of Psychology University of Waterloo(滑铁卢大学心理学系)

Nolan P Shaw, P Michael Furlong, Britt Anderson, Jeff Orchard

AI总结:

本文首次将范畴论应用于向量符号架构,将向量推广为余预层,并把 VSA 操作形式化为外部张量积的右 Kan 扩展,通过示例验证了该形式化并为 VSA 新设计提供思路。

摘要:

当前,联结主义的机器学习方法,即神经网络,正受到广泛关注。然而,这些方法需要大量数据,并且生成的模型对人类而言不可解释。向量符号架构(VSAs)是一种替代框架,它与神经网络及基于梯度的学习兼容,同时显式地对组合性进行建模。VSAs 是高维向量表示上的一族代数,起源于认知科学中统一神经处理与人类所进行的符号推理的需求。虽然机器学习方法已从范畴论分析中获益,但 VSAs 尚未获得类似的处理。在本文中,我们首次尝试将范畴论应用于 VSAs。具体而言,我们将向量推广到余预层,并将 VSA 操作描述为外部张量积的右 Kan 扩展。这一形式化过程包含一个证明,表明此类情况下的右 Kan 扩展可以表示为简单的逐元素操作。我们通过连接当前 VSA 实现的具体示例验证了该形式化,同时为 VSAs 提出了新的可能设计。

英文摘要:

Connectionist approaches to machine learning, \emph{i.e.} neural networks, are enjoying a considerable vogue right now. However, these methods require large volumes of data and produce models that are uninterpretable to humans. An alternative framework that is compatible with neural networks and gradient-based learning, but explicitly models compositionality, is Vector Symbolic Architectures (VSAs). VSAs are a family of algebras on high-dimensional vector representations. They arose in cognitive science from the need to unify neural processing and the kind of symbolic reasoning that humans perform. While machine learning methods have benefited from category-theoretical analyses, VSAs have not yet received similar treatment. In this paper, we present a first attempt at applying category theory to VSAs. Specifically, We generalise from vectors to co-presheaves, and describe VSA operations as the right Kan extensions of the external tensor product. This formalisation involves a proof that the right Kan extension in such cases can be expressed as simple, element-wise operations. We validate our formalisation with worked examples that connect to current VSA implementations, while suggesting new possible designs for VSAs.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑