arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2404.08458stat.MLcs.AIcs.LG
收藏

论神经符号学习中的独立性假设

On the Independence Assumption in Neurosymbolic Learning

School of Informatics, University of Edinburgh(爱丁堡大学信息学院)

Emile van Krieken, Pasquale Minervini, Edoardo M. Ponti, Antonio Vergari

AI总结:

本文批判神经符号学习中符号概率条件独立的假设,证明其导致过度自信、难以优化且无法量化不确定性,为设计更具表达力的神经符号概率模型提供理论基础。

摘要:

最先进的神经符号学习系统使用概率推理来引导神经网络做出符合符号逻辑约束的预测。许多此类系统假设所考虑符号的概率在给定输入的情况下是条件独立的,以简化学习和推理。我们研究并批判了这一假设,强调它如何阻碍优化并妨碍不确定性量化。我们证明,损失函数会使条件独立的神经网络偏向于对其预测过度自信。因此,它们无法表示多个有效选项上的不确定性。此外,我们证明这些损失函数难以优化:它们是非凸的,且其最小值通常高度不连通。我们的理论分析为取代条件独立性假设和设计更具表达力的神经符号概率模型奠定了基础。

英文摘要:

State-of-the-art neurosymbolic learning systems use probabilistic reasoning to guide neural networks towards predictions that conform to logical constraints over symbols. Many such systems assume that the probabilities of the considered symbols are conditionally independent given the input to simplify learning and reasoning. We study and criticise this assumption, highlighting how it can hinder optimisation and prevent uncertainty quantification. We prove that loss functions bias conditionally independent neural networks to become overconfident in their predictions. As a result, they are unable to represent uncertainty over multiple valid options. Furthermore, we prove that these loss functions are difficult to optimise: they are non-convex, and their minima are usually highly disconnected. Our theoretical analysis gives the foundation for replacing the conditional independence assumption and designing more expressive neurosymbolic probabilistic models.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑