通过准符号抽象改进链式推理
Improving Chain-of-Thought Reasoning via Quasi-Symbolic Abstractions
AI总结：
本文提出QuaSAR方法，通过准符号解释引导LLM在更高抽象层面推理，提升小模型的推理能力，实验显示在自然语言和符号推理任务中准确率提升8%。
摘要：
链式推理（CoT）是大型语言模型（LLMs）中常见的推理策略，通过将复杂任务分解为中间推理步骤。然而，CoT生成的解释容易受到内容偏见的影响，影响其鲁棒性和准确性。为缓解现有局限，近期工作提出了结合逻辑形式化与外部符号求解器的方法。然而，完全符号化方法存在瓶颈，即需要将自然语言完全翻译为形式语言，影响效率和灵活性。为此，本文研究了在不完全形式化的情况下分离内容与逻辑推理的方法。特别地，我们提出了QuaSAR（准符号抽象推理），一种CoT的变体，通过准符号解释引导LLM在更高抽象层面操作。我们的框架利用LLM仅能形式化相关变量和谓词的能力，使符号元素与自然语言共存。我们展示了QuaSAR在上下文学习和构建演示中的影响，以提升小模型的推理能力。实验表明，准符号抽象可以将基于CoT的方法的准确率提升高达8%，在自然语言（如MMLU-Redux）和符号推理任务（如GSM-Symbolic）的挑战性对抗变体上增强了鲁棒性和一致性。
英文摘要：
Chain-of-Though (CoT) represents a common strategy for reasoning in Large Language Models (LLMs) by decomposing complex tasks into intermediate inference steps. However, explanations generated via CoT are susceptible to content biases that negatively affect their robustness and faithfulness. To mitigate existing limitations, recent work has proposed using logical formalisms coupled with external symbolic solvers. However, fully symbolic approaches possess the bottleneck of requiring a complete translation from natural language to formal languages, a process that affects efficiency and flexibility. To achieve a trade-off, this paper investigates methods to disentangle content from logical reasoning without a complete formalisation. In particular, we present QuaSAR (for Quasi-Symbolic Abstract Reasoning), a variation of CoT that guides LLMs to operate at a higher level of abstraction via quasi-symbolic explanations. Our framework leverages the capability of LLMs to formalise only relevant variables and predicates, enabling the coexistence of symbolic elements with natural language. We show the impact of QuaSAR for in-context learning and for constructing demonstrations to improve the reasoning capabilities of smaller models. Our experiments show that quasi-symbolic abstractions can improve CoT-based methods by up to 8% accuracy, enhancing robustness and consistency on challenging adversarial variations on both natural language (i.e. MMLU-Redux) and symbolic reasoning tasks (i.e., GSM-Symbolic).
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
