嵌入作为认知状态：使用池化算子积累知识的局限性
Embeddings as Epistemic States: Limitations on the Use of Pooling Operators for Accumulating Knowledge
AI总结：
本文研究了标准池化算子满足“认知池化原则”的条件，发现其依赖高维和特定约束，并指出这些约束导致线性评分函数大多无法验证命题公式，从而解释了GNN在推理任务中的局限性。
摘要：
各种神经网络架构依赖池化算子来聚合来自不同来源的信息。在此类情境中，通常隐含地假设向量编码了认知状态，即向量捕获了关于某些感兴趣性质所获得的证据，并且对这些向量进行池化会得到一个结合了这些证据的向量。针对多种标准池化算子，我们研究了它们在何种条件下符合这一想法，我们将其称为认知池化原则。我们发现，所有被考虑的池化算子都能满足认知池化原则，但这仅在嵌入具有足够高维度时成立，且对于大多数池化算子，还要求嵌入满足特定约束（例如具有非负坐标）。我们进一步表明，这些约束对嵌入在实践中如何使用具有重要影响。具体而言，我们发现当满足认知池化原则时，在大多数情况下无法使用线性评分函数来验证命题公式的满足性，但有两个例外：（i）具有上界嵌入的 max-pooling 和（ii）具有非负嵌入的 Hadamard pooling。这一发现有助于澄清，除其他外，为何 Graph Neural Networks 在推理任务中有时表现不佳。最后，我们还研究了认知池化原则向加权认知状态的扩展，这在非单调推理的背景下很重要，其中 max-pooling 成为最合适的算子。
英文摘要：
Various neural network architectures rely on pooling operators to aggregate information coming from different sources. It is often implicitly assumed in such contexts that vectors encode epistemic states, i.e. that vectors capture the evidence that has been obtained about some properties of interest, and that pooling these vectors yields a vector that combines this evidence. We study, for a number of standard pooling operators, under what conditions they are compatible with this idea, which we call the epistemic pooling principle. While we find that all the considered pooling operators can satisfy the epistemic pooling principle, this only holds when embeddings are sufficiently high-dimensional and, for most pooling operators, when the embeddings satisfy particular constraints (e.g. having non-negative coordinates). We furthermore show that these constraints have important implications on how the embeddings can be used in practice. In particular, we find that when the epistemic pooling principle is satisfied, in most cases it is impossible to verify the satisfaction of propositional formulas using linear scoring functions, with two exceptions: (i) max-pooling with embeddings that are upper-bounded and (ii) Hadamard pooling with non-negative embeddings. This finding helps to clarify, among others, why Graph Neural Networks sometimes under-perform in reasoning tasks. Finally, we also study an extension of the epistemic pooling principle to weighted epistemic states, which are important in the context of non-monotonic reasoning, where max-pooling emerges as the most suitable operator.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
