自适应选择符号语言以提升LLM逻辑推理能力
Adaptive Selection of Symbolic Languages for Improving LLM Logical Reasoning
AI总结：
本文提出一种自适应选择符号语言的方法，通过在翻译前为不同自然语言逻辑推理问题匹配最优符号语言（一阶逻辑、逻辑编程或布尔可满足性），显著提升了LLM的逻辑推理性能。
摘要：
大型语言模型（LLMs）在复杂逻辑推理方面仍然存在困难。虽然先前的工作取得了显著改进，但其性能高度依赖于将自然语言（NL）问题翻译为符号语言（SL）的正确性。尽管许多工作致力于提高这种翻译的准确性，但它们仅考虑了SL与NL在语义上的相似性，却忽略了另一个关键的影响因素，即目标SL类型本身的选择。例如，一阶逻辑语言专攻涉及范畴三段论和复杂量词的逻辑推理，而布尔可满足性形式主义则擅长表示如部分问题等约束满足问题。据我们所知，这是第一篇提出并验证不同NL逻辑推理问题对应不同最优SL形式化进行翻译的论文。基于此，我们提出了一种方法，通过在翻译前为每个问题自适应选择最合适的SL来提升LLMs的逻辑推理性能。具体而言，我们利用LLMs在一阶逻辑、逻辑编程和布尔可满足性之间选择目标SL，然后将NL问题翻译为目标SL表达式，并采用相应的逻辑求解器推导最终答案。在基准测试上的实验结果表明，我们的自适应选择方法显著优于将所有问题翻译为单一SL以及随机选择SL的做法。在这些基准的混合数据集上，我们的方法达到了96%的准确率，相比排名第二的一阶逻辑翻译准确率提升了25%的性能。
英文摘要：
Large Language Models (LLMs) still struggle with complex logical reasoning. While previous works achieve remarkable improvements, their performance is highly dependent on the correctness of translating natural language (NL) problems into a symbolic language (SL). Though numerous works focusing on improving this translation accuracy, they only consider the similarity between the meaning of SL and NL, overlooking another crucial influencing factor, the selection of the target SL type itself. For example, first-order logic language specializes in logical reasoning with categorical syllogisms and complex quantifiers, while Boolean satisfiability formalism excels at representing constraint satisfaction like partial problems. To our knowledge, this is the first paper to claim and verify that different NL logical reasoning problem corresponds to different optimal SL formalization for translation. Based on this, we propose a methods to improve the logical reasoning performance of LLMs by adaptively selecting the most suitable SL for each problem prior to translation. Specifically, we leverage LLMs to select the target SL among first-order logic, logic programming and Boolean satisfiability and then translate the problem in NL to target SL expressions as well as employ the corresponding logical solver to derive the final answer. Experimental results on benchmarks show that our adaptive selection method significantly outperforms translating all into single SL and randomly selecting the SL. On a mixed dataset of these benchmarks, our approach achieves 96% accuracy, which improving performance by 25% compared to the second highest accuracy from the first-order logic translation.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
