大语言模型中的短路径提示：分析推理不稳定性及鲁棒性能解决方案
Short-Path Prompting in LLMs: Analyzing Reasoning Instability and Solutions for Robust Performance
AI总结：
本文针对用户短路径提示与LLMs的CoT推理需求冲突导致推理性能下降且不稳定的问题，提出指令引导和微调两种方法，有效提升了推理精度，为指令遵循与推理准确性的权衡提供了见解。
摘要：
近年来，大语言模型（LLMs）的推理能力取得了显著进步，这在很大程度上归功于思维链（CoT）方法，该方法允许模型在得出最终答案前生成中间推理步骤。在这些进展的基础上，最先进的LLMs经过指令微调，在回答推理相关问题时能提供冗长且详细的CoT路径。然而，人类天生是认知吝啬者，会提示语言模型给出相当简短的响应，从而与CoT推理产生显著冲突。本文深入研究了当用户提供短路径提示时，LLMs的推理性能如何变化。结果和分析表明，语言模型无需显式CoT提示即可有效且鲁棒地推理，但在短路径提示下，即使在小学水平的问题上，LLMs的推理能力也会显著下降并变得不稳定。为解决这一问题，我们提出两种方法：指令引导方法和微调方法，两者均旨在有效管理该冲突。实验结果表明，两种方法均实现了高精度，为当前模型中指令遵循与推理准确性之间的权衡提供了见解。
英文摘要：
Recent years have witnessed significant progress in large language models' (LLMs) reasoning, which is largely due to the chain-of-thought (CoT) approaches, allowing models to generate intermediate reasoning steps before reaching the final answer. Building on these advances, state-of-the-art LLMs are instruction-tuned to provide long and detailed CoT pathways when responding to reasoning-related questions. However, human beings are naturally cognitive misers and will prompt language models to give rather short responses, thus raising a significant conflict with CoT reasoning. In this paper, we delve into how LLMs' reasoning performance changes when users provide short-path prompts. The results and analysis reveal that language models can reason effectively and robustly without explicit CoT prompts, while under short-path prompting, LLMs' reasoning ability drops significantly and becomes unstable, even on grade-school problems. To address this issue, we propose two approaches: an instruction-guided approach and a fine-tuning approach, both designed to effectively manage the conflict. Experimental results show that both methods achieve high accuracy, providing insights into the trade-off between instruction adherence and reasoning accuracy in current models.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
