arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.03136cs.CL
收藏

语言模型中的审慎推理:基于精确世界模型的结构感知规划

Deliberate Reasoning in Language Models as Structure-Aware Planning with an Accurate World Model

Georgia Institute of Technology(佐治亚理工学院);Cisco Research(思科研究)

Siheng Xiong, Ali Payani, Yuan Yang, Faramarz Fekri

AI总结:

针对语言模型复杂推理任务中现有CoT方法的一致性与验证问题,提出SWAP框架,结合蕴含图结构验证、多样性探索与对比排序,在多类推理基准上显著优于基础模型及现有方法。

摘要:

增强语言模型(LMs)的推理能力仍是一项关键挑战,尤其对于需要复杂多步决策的任务——现有思维链(CoT)方法在这些任务中难以保证一致性和可验证性。本文提出一种名为“基于精确世界模型的结构感知规划(SWAP)”的新型推理框架,将结构化知识表示与学习型规划相集成。与单纯依赖自然语言推理的现有方法不同,SWAP利用蕴含图编码结构依赖关系,实现中间步骤的符号验证。为系统构建和更新该图,SWAP采用策略模型提出候选扩展,并通过世界模型预测结构更新。为提升准确性,世界模型生成多个备选更新,再由判别器基于合理性重排序。为鼓励多样化探索,我们引入基于多样性的建模(DM),从原始策略分布中移除已采样候选后,从剩余概率质量中采样新候选。此外,SWAP通过对比排序(CR)提升判别准确性,该方法在提示词内直接比较候选,并融入元知识以改善排序质量。我们在数学推理、逻辑推理和编码任务等多种推理密集型基准上评估SWAP,大量实验表明SWAP显著优于基础模型,并持续超越现有推理方法。

英文摘要:

Enhancing the reasoning capabilities of language models (LMs) remains a key challenge, especially for tasks that require complex, multi-step decision-making where existing Chain-of-Thought (CoT) approaches struggle with consistency and verification. In this paper, we propose a novel reasoning framework, referred to as Structure-aware Planning with an Accurate World Model (SWAP), that integrates structured knowledge representation with learned planning. Unlike prior methods that rely purely on natural language reasoning, SWAP leverages entailment graphs to encode structured dependencies and enable symbolic verification of intermediate steps. To systematically construct and update the graph, SWAP employs a policy model to propose candidate expansions and a world model to predict structural updates. To improve accuracy, the world model generates multiple alternative updates, and a discriminator re-ranks them based on plausibility. To encourage diverse exploration, we introduce Diversity-based Modelling (DM), which samples candidates from the remaining probability mass after removing previously sampled candidates from the original policy distribution. Additionally, SWAP improves the discrimination accuracy through Contrastive Ranking (CR), which directly compares candidates within prompts and incorporates meta-knowledge to improve ranking quality. We evaluate SWAP across diverse reasoning-intensive benchmarks including math reasoning, logical reasoning, and coding tasks. Extensive experiments demonstrate that SWAP significantly improves upon the base models and consistently outperforms existing reasoning methods.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑