面向神经符号学习的约束引导扩散推理器
Constraints-Guided Diffusion Reasoner for Neuro-Symbolic Learning
AI总结：
本文提出约束引导扩散推理器，将扩散模型形式化为马尔可夫决策过程并用改进PPO微调，通过两阶段训练在数独、迷宫等符号推理任务中实现高准确率与逻辑一致性。
摘要：
使神经网络能够学习复杂逻辑约束并完成符号推理是一项关键挑战。弥合这一差距通常需要引导神经网络的输出分布更接近符号约束。尽管扩散模型已在多个领域展现出卓越的生成能力，我们采用这一强大架构来执行神经符号学习并解决逻辑谜题。我们的基于扩散的流程采用两阶段训练策略：第一阶段专注于培养基本推理能力，第二阶段则强调对逻辑约束的系统性学习。为了在第二阶段对神经输出施加硬约束，我们将扩散推理器形式化为马尔可夫决策过程，并创新性地使用改进的近端策略优化算法对其进行微调。我们利用源自神经输出逻辑一致性的基于规则的奖励信号，并采用灵活策略来优化扩散推理器的策略。我们在一些经典符号推理基准上评估了我们的方法，包括数独、迷宫、路径规划和偏好学习。实验结果表明，我们的方法在神经网络中实现了出色的准确率和逻辑一致性。
英文摘要：
Enabling neural networks to learn complex logical constraints and fulfill symbolic reasoning is a critical challenge. Bridging this gap often requires guiding the neural network's output distribution to move closer to the symbolic constraints. While diffusion models have shown remarkable generative capability across various domains, we employ the powerful architecture to perform neuro-symbolic learning and solve logical puzzles. Our diffusion-based pipeline adopts a two-stage training strategy: the first stage focuses on cultivating basic reasoning abilities, while the second emphasizes systematic learning of logical constraints. To impose hard constraints on neural outputs in the second stage, we formulate the diffusion reasoner as a Markov decision process and innovatively fine-tune it with an improved proximal policy optimization algorithm. We utilize a rule-based reward signal derived from the logical consistency of neural outputs and adopt a flexible strategy to optimize the diffusion reasoner's policy. We evaluate our methodology on some classical symbolic reasoning benchmarks, including Sudoku, Maze, pathfinding and preference learning. Experimental results demonstrate that our approach achieves outstanding accuracy and logical consistency among neural networks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
