实例自适应的零样本思维链提示
Instance-adaptive Zero-shot Chain-of-Thought Prompting
AI总结：
本研究针对单一任务级零样本CoT提示适配性不足的问题，从信息流视角揭示推理机制，提出实例自适应提示策略IAP，在多个模型和推理任务上实现了性能提升。
摘要：
零样本思维链（Chain-of-Thought, CoT）提示是提升大语言模型（LLM）在现实推理任务中表现的一种简单有效的策略。然而，在所有实例上统一应用单一任务级提示的效果存在固有局限，因为一条提示不可能适配所有实例，更合适的方法应当细致考量提示与每个实例之间的交互。本研究提出一种实例自适应提示算法，作为零样本CoT推理的替代方案，能够自适应区分优质和劣质提示。具体而言，我们首先从信息流视角对LLM展开分析，以探究零样本CoT推理的内在机制，研究发现，从问题到提示的信息流以及从问题到推理依据的信息流共同对推理结果的影响最大。我们注意到，更优的零样本CoT推理需要提示从问题中获取语义信息，随后推理依据既要直接从问题中聚合充足信息，也要通过提示间接获取信息。反之，缺少其中任意一环都可能导致推理效果不佳。基于这一发现，我们进一步提出了用于零样本CoT推理的实例自适应提示策略（Instance-Adaptive Prompting, IAP）。我们使用LLaMA-2、LLaMA-3和Qwen在数学、逻辑和常识推理任务（如GSM8K、MMLU、因果判断）上开展了实验，结果显示该方法取得了一致性的性能提升，证明实例自适应零样本CoT提示的表现优于其他采用精心设计提示或复杂流程的任务级方法，也体现了我们关于零样本CoT推理机制的发现的重要意义。
英文摘要：
Zero-shot Chain-of-Thought (CoT) prompting emerges as a simple and effective strategy for enhancing the performance of large language models (LLMs) in real-world reasoning tasks. Nonetheless, the efficacy of a singular, task-level prompt uniformly applied across the whole of instances is inherently limited since one prompt cannot be a good partner for all, a more appropriate approach should consider the interaction between the prompt and each instance meticulously. This work introduces an instance-adaptive prompting algorithm as an alternative zero-shot CoT reasoning scheme by adaptively differentiating good and bad prompts. Concretely, we first employ analysis on LLMs through the lens of information flow to detect the mechanism under zero-shot CoT reasoning, in which we discover that information flows from question to prompt and question to rationale jointly influence the reasoning results most. We notice that a better zero-shot CoT reasoning needs the prompt to obtain semantic information from the question then the rationale aggregates sufficient information from the question directly and via the prompt indirectly. On the contrary, lacking any of those would probably lead to a bad one. Stem from that, we further propose an instance-adaptive prompting strategy (IAP) for zero-shot CoT reasoning. Experiments conducted with LLaMA-2, LLaMA-3, and Qwen on math, logic, and commonsense reasoning tasks (e.g., GSM8K, MMLU, Causal Judgement) obtain consistent improvement, demonstrating that the instance-adaptive zero-shot CoT prompting performs better than other task-level methods with some curated prompts or sophisticated procedures, showing the significance of our findings in the zero-shot CoT reasoning mechanism.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
