Code Prompting：一种面向大语言模型复杂推理的神经符号方法
Code Prompting: a Neural Symbolic Method for Complex Reasoning in Large Language Models
AI总结：
针对大语言模型自然语言中间推理步骤的缺陷，提出以代码为中间步骤的神经符号提示方法Code Prompting，在7个推理基准上优于CoT提示，并分析了其优势与影响因素。
摘要：
大语言模型（LLMs）规模不断扩大，借助各类提示词方法已能解锁广泛的复杂推理任务。然而，当前的提示词方法通过生成自然语言中间步骤辅助推理，可能导致任务拆解不完善与逻辑混淆。为缓解这些局限，我们探究了Code Prompting（代码提示）——一种兼具零样本和少样本版本的神经符号提示方法，该方法以生成代码作为推理中间步骤。我们在7个涉及符号推理与算术推理的常用基准上开展实验，Code Prompting的表现整体优于chain-of-thought（CoT，思维链）提示。为进一步理解Code Prompting的性能与局限，我们开展了大量消融实验与错误分析，明确了符号提示相较于自然语言提示的若干独有优势。我们还尝试将Code Prompting与CoT提示集成以结合二者优势，最后通过实验验证了代码注释及其位置对Code Prompting效果的影响。
英文摘要：
Large language models (LLMs) have scaled up to unlock a wide range of complex reasoning tasks with the aid of various prompting methods. However, current prompting methods generate natural language intermediate steps to help reasoning, which can cause imperfect task reduction and confusion. To mitigate such limitations, we explore code prompting, a neural symbolic prompting method with both zero-shot and few-shot versions which triggers code as intermediate steps. We conduct experiments on 7 widely-used benchmarks involving symbolic reasoning and arithmetic reasoning. Code prompting generally outperforms chain-of-thought (CoT) prompting. To further understand the performance and limitations of code prompting, we perform extensive ablation studies and error analyses, and identify several exclusive advantages of using symbolic promptings compared to natural language. We also consider the ensemble of code prompting and CoT prompting to combine the strengths of both. Finally, we show through experiments how code annotations and their locations affect code prompting.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
