一种基于复杂性的组合性理论
A Complexity-Based Theory of Compositionality
AI总结：
本文提出基于算法信息论的表示组合性形式化定义，要求表示具有表达力、可重组为符号序列且语义映射简单，并通过实验验证其能统一AI与认知科学中的组合性直觉。
摘要：
组合性被认为是智能的基础。在人类中，它是思维、语言和更高层次推理结构的基础。在人工智能中，组合性表示可以支持一种强大的分布外泛化形式，使模型能够系统性地适应已知概念的新组合。然而，尽管我们对什么是组合性有很强的直觉，但缺乏令人满意的、可度量且数学化的正式定义。在此，我们提出了这样一个定义，称之为表示组合性，它解释并扩展了我们对组合性的直觉。该定义在概念上简单、定量、以算法信息论为基础，并适用于任何表示。直观地说，表示组合性表明组合性表示满足三个性质。第一，它必须具有表达力。第二，必须能够将该表示重新描述为具有可重组部分的离散符号序列的函数，类似于自然语言中的句子。第三，将这些符号序列与表示联系起来的函数，类似于自然语言中的语义，必须是简单的。通过在合成数据和真实世界数据上的实验，我们验证了我们的组合性定义，并展示了它如何统一来自人工智能和认知科学文献中的不同直觉。我们还表明，表示组合性虽然在理论上不可计算，但可以使用标准深度学习工具轻松估计。我们希望我们的定义能够启发设计新颖的、理论驱动且能更好地捕捉组合性思维机制的模型。我们的代码可在 https://github.com/EricElmoznino/complexity_compositionality 获取。
英文摘要：
Compositionality is believed to be fundamental to intelligence. In humans, it underlies the structure of thought, language, and higher-level reasoning. In AI, compositional representations can enable a powerful form of out-of-distribution generalization, in which a model systematically adapts to novel combinations of known concepts. However, while we have strong intuitions about what compositionality is, we lack satisfying formal definitions for it that are measurable and mathematical. Here, we propose such a definition, which we call representational compositionality, that accounts for and extends our intuitions about compositionality. The definition is conceptually simple, quantitative, grounded in algorithmic information theory, and applicable to any representation. Intuitively, representational compositionality states that a compositional representation satisfies three properties. First, it must be expressive. Second, it must be possible to re-describe the representation as a function of discrete symbolic sequences with re-combinable parts, analogous to sentences in natural language. Third, the function that relates these symbolic sequences to the representation, analogous to semantics in natural language, must be simple. Through experiments on both synthetic and real world data, we validate our definition of compositionality and show how it unifies disparate intuitions from across the literature in both AI and cognitive science. We also show that representational compositionality, while theoretically intractable, can be readily estimated using standard deep learning tools. We hope that our definition can inspire the design of novel, theoretically-driven models that better capture the mechanisms of compositional thought. We make our code available at https://github.com/EricElmoznino/complexity_compositionality.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
