AbstRaL: 通过强化抽象思维增强大语言模型的推理能力
AbstRaL: Augmenting LLMs' Reasoning by Reinforcing Abstract Thinking
AI总结：
AbstRaL通过强化学习提升LLM的抽象推理能力，有效缓解GSM扰动基准测试中的性能下降，并提升模型在OD数学和一般推理任务上的泛化能力。
摘要：
最近的研究表明，大型语言模型（LLMs），尤其是较小的模型，在小学数学（GSM）推理中往往缺乏鲁棒性。特别是，当面对分布偏移时，如数值或名义变量的变化，或插入干扰性条款时，它们的性能会下降。可能的解决策略是生成合成数据，以进一步
英文摘要：
Recent studies have shown that large language models (LLMs), especially smaller ones, often lack robustness in grade school math (GSM) reasoning. In particular, they tend to experience performance drops when faced with distribution shifts, such as changes to numerical or nominal variables, or insertions of distracting clauses. A possible strategy to address this involves generating synthetic data to further "instantiate" reasoning problems on potential variations. In this work, we instead focus on the strategy of "abstracting" reasoning problems. This not only helps counteract distribution shifts but also facilitates the connection to symbolic tools for deriving solutions. Focusing on GSM, we find that this abstraction process is better acquired through reinforcement learning (RL) than just supervised fine-tuning, which often fails to produce faithful abstractions. Our method, AbstRaL -- which promotes abstract reasoning in LLMs using RL on granular abstraction data -- significantly mitigates performance degradation on recent GSM perturbation benchmarks. Besides, improving GSM robustness via AbstRaL is shown to also implicitly benefit LLMs' capabilities on OOD mathematical and general reasoning tasks, indicating that abstract thinking broadly enables better generalizability.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
