arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2402.14047cs.LGcs.AI
收藏

用于神经符号集成的简单有效迁移学习方法

Simple and Effective Transfer Learning for Neuro-Symbolic Integration

Fondazione Bruno Kessler(布鲁诺·凯塞勒基金会);TU Wien(维也纳工业大学)

Alessandro Daniele, Tommaso Campari, Sagar Malhotra, Luciano Serafini

AI总结:

针对神经符号集成方法收敛慢、复杂感知任务学习难等问题,提出先预训练神经模型再迁移感知部分权重的训练策略,在多种SOTA方法和数据集上实现了性能的持续提升。

摘要:

近年来,深度学习(DL)技术已取得显著成效,但其泛化能力与推理任务执行能力仍存在挑战。神经符号集成(Neuro-Symbolic Integration, NeSy)是解决该问题的潜在方案,其将神经方法与符号推理相结合。这类方法大多利用神经网络将感知映射为符号,再通过逻辑推理器预测下游任务的输出,与全神经架构相比具备更优的泛化能力,但也存在收敛速度慢、复杂感知任务学习困难、易收敛至局部最小值等问题。本文提出一种简单有效的方法以改善上述问题,核心思路是先在下游任务上预训练一个神经模型,再通过迁移学习在同一任务上训练NeSy模型,其中感知部分的权重由预训练网络注入。本研究的关键发现是:神经网络完全能够学习从感知到符号的映射,仅在符号层面存在泛化失效问题。我们在多种当前最优(SOTA)的NeSy方法和数据集上测试了该训练策略,结果表明其在上述问题上均取得了持续的性能提升。

英文摘要:

Deep Learning (DL) techniques have achieved remarkable successes in recent years. However, their ability to generalize and execute reasoning tasks remains a challenge. A potential solution to this issue is Neuro-Symbolic Integration (NeSy), where neural approaches are combined with symbolic reasoning. Most of these methods exploit a neural network to map perceptions to symbols and a logical reasoner to predict the output of the downstream task. These methods exhibit superior generalization capacity compared to fully neural architectures. However, they suffer from several issues, including slow convergence, learning difficulties with complex perception tasks, and convergence to local minima. This paper proposes a simple yet effective method to ameliorate these problems. The key idea involves pretraining a neural model on the downstream task. Then, a NeSy model is trained on the same task via transfer learning, where the weights of the perceptual part are injected from the pretrained network. The key observation of our work is that the neural network fails to generalize only at the level of the symbolic part while being perfectly capable of learning the mapping from perceptions to symbols. We have tested our training strategy on various SOTA NeSy methods and datasets, demonstrating consistent improvements in the aforementioned problems.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑