重新思考大语言模型中的推理：超越上下文学习与思维链的神经符号局部 RetoMaton
Rethinking Reasoning in LLMs: Neuro-Symbolic Local RetoMaton Beyond ICL and CoT
AI总结：
本文提出用局部任务自适应加权有限自动机替代 RetoMaton 的全局数据存储，以克服 CoT 和 ICL 推理不稳定、不可解释的问题，并在多个推理任务上验证其性能提升与可追溯性。
摘要：
基于提示的推理策略，如思维链（Chain-of-Thought, CoT）和上下文学习（In-Context Learning, ICL），已被广泛用于激发大语言模型（LLMs）的推理能力。然而，这些方法依赖于脆弱且隐式的机制，常常在不同随机种子、格式或微小的提示变化下产生不一致的输出，使其在需要稳定、可解释推理的任务中从根本上不可靠。相比之下，基于自动机的神经符号框架（如 RetoMaton）通过将检索建立在具有确定性转移的符号记忆中，提供了一种更具结构性和可信度的替代方案。在本工作中，我们扩展了 RetoMaton，将其全局数据存储替换为一个局部的、任务自适应的加权有限自动机（Weighted Finite Automaton, WFA），该自动机直接从外部领域语料库构建。这种局部自动机结构促进了稳健的、上下文感知的检索，同时保留了符号可追溯性和较低的推理开销。与以不透明方式纠缠上下文和记忆的提示方法不同，我们的方法利用 WFA 的显式结构来提供可验证和模块化的检索行为，使其更适合领域迁移和互操作性。我们在两个预训练大语言模型 LLaMA-3.2-1B 和 Gemma-3-1B-PT 上，跨三个推理任务评估了这一局部 RetoMaton 变体：TriviaQA（阅读理解）、GSM8K（多步数学）和 MMLU（领域知识）。与基础模型和基于提示的方法相比，用局部 RetoMaton 增强这些设置能够持续提升性能，同时实现透明且可复现的检索动态。我们的结果凸显了通过轻量级、自动机引导的记忆，在现代大语言模型中迈向可信赖的符号推理的有前景的转变。
英文摘要：
Prompt-based reasoning strategies such as Chain-of-Thought (CoT) and In-Context Learning (ICL) have become widely used for eliciting reasoning capabilities in large language models (LLMs). However, these methods rely on fragile, implicit mechanisms often yielding inconsistent outputs across seeds, formats, or minor prompt variations making them fundamentally unreliable for tasks requiring stable, interpretable reasoning. In contrast, automata-based neuro-symbolic frameworks like RetoMaton offer a more structured and trustworthy alternative by grounding retrieval in symbolic memory with deterministic transitions. In this work, we extend RetoMaton by replacing its global datastore with a local, task-adaptive Weighted Finite Automaton (WFA), constructed directly from external domain corpora. This local automaton structure promotes robust, context-aware retrieval while preserving symbolic traceability and low inference overhead. Unlike prompting, which entangles context and memory in opaque ways, our approach leverages the explicit structure of WFAs to provide verifiable and modular retrieval behavior, making it better suited for domain transfer and interoperability. We evaluate this local RetoMaton variant on two pretrained LLMs LLaMA-3.2-1B and Gemma-3-1B-PT across three reasoning tasks: TriviaQA (reading comprehension), GSM8K (multi-step math), and MMLU (domain knowledge). Compared to the base model and prompting-based methods, augmenting these setups with local RetoMaton consistently improves performance while enabling transparent and reproducible retrieval dynamics. Our results highlight a promising shift toward trustworthy, symbolic reasoning in modern LLMs via lightweight, automaton-guided memory.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
