基于DSPy的神经符号流水线以增强大语言模型的空间推理能力
Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs
AI总结：
本研究提出基于DSPy的LLM与ASP迭代反馈神经符号框架，结合模块化解析、迭代反馈与鲁棒错误处理，在StepGame和SparQA数据集上显著提升空间推理准确率，兼具泛化性与轻量替代方案。
摘要：
大语言模型（LLM）已在各类任务中展现出卓越能力，但往往在空间推理方面存在困难。本文提出一种新型神经符号框架，通过LLM与回答集编程（Answer Set Programming，ASP）之间的迭代反馈增强LLM的空间推理能力。我们在两个基准数据集StepGame和SparQA上评估所提方法，实现了三种不同策略：（1）直接提示基线，（2）Facts+Rules提示，（3）基于DSPy的带迭代优化的LLM+ASP流水线。实验结果表明，LLM+ASP流水线显著优于基线方法，在StepGame上达到平均82%的准确率，在SparQA上达到69%，较直接提示分别提升40-50%和8-15%。其成功源于三项关键创新：（1）通过模块化流水线有效分离语义解析与逻辑推理；（2）LLM与ASP求解器之间的迭代反馈机制提升了程序生成率；（3）鲁棒的错误处理机制可应对解析、实例化与求解失败问题。此外，我们提出Facts+Rules作为轻量替代方案，在复杂的SparQA数据集上取得了可比性能，同时降低了计算开销。我们在不同LLM架构（Deepseek、Llama3-70B、GPT-4.0 mini）上的分析证明了该框架的泛化性，并揭示了实现复杂度与推理能力之间的权衡，为开发更具可解释性与可靠性的AI系统做出贡献。
英文摘要：
Large Language Models (LLMs) have demonstrated remarkable capabilities across various tasks, yet they often struggle with spatial reasoning. This paper presents a novel neural-symbolic framework that enhances LLMs' spatial reasoning abilities through iterative feedback between LLMs and Answer Set Programming (ASP). We evaluate our approach on two benchmark datasets: StepGame and SparQA, implementing three distinct strategies: (1) direct prompting baseline, (2) Facts+Rules prompting, and (3) DSPy-based LLM+ASP pipeline with iterative refinement. Our experimental results demonstrate that the LLM+ASP pipeline significantly outperforms baseline methods, achieving an average 82% accuracy on StepGame and 69% on SparQA, marking improvements of 40-50% and 8-15% respectively over direct prompting. The success stems from three key innovations: (1) effective separation of semantic parsing and logical reasoning through a modular pipeline, (2) iterative feedback mechanism between LLMs and ASP solvers that improves program rate, and (3) robust error handling that addresses parsing, grounding, and solving failures. Additionally, we propose Facts+Rules as a lightweight alternative that achieves comparable performance on complex SparQA dataset, while reducing computational overhead.Our analysis across different LLM architectures (Deepseek, Llama3-70B, GPT-4.0 mini) demonstrates the framework's generalizability and provides insights into the trade-offs between implementation complexity and reasoning capability, contributing to the development of more interpretable and reliable AI systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
