面向大语言模型的提示草图方法
Prompt Sketching for Large Language Models
AI总结：
针对大语言模型顺序提示中间响应脱节冗长的问题，提出提示草图范式，通过调整解码流程优化模板似然，零样本下在多数基准任务上优于现有方案，并发布相关草图和开源库。
摘要：
当前许多面向大语言模型（LLM）的提示策略会按顺序多次查询模型——先生成中间结果，再得出最终答案。但使用这类方法时，解码器和模型都无法感知后续可能出现的提示，导致中间响应内容脱节且过于冗长。针对这一问题，本文提出提示草图（prompt sketching）这一全新提示范式：在该范式下，大语言模型的响应不再仅是补全提示，而是要预测模板中多个变量的取值。通过这种方式，草图方法能让用户对生成过程拥有更强的掌控力，例如可通过中间指令提供推理框架，从而提升整体输出质量。要让现有自回归模型支持草图功能，核心思路是调整解码流程，使其在文本生成过程中同时对后续指令进行评分，从而在推理阶段优化整体模板的似然度。实验结果表明，在零样本设置下，提示草图在8项大语言模型基准任务（包括状态追踪、算术推理和通用问答）中的7项上，表现优于现有的顺序提示方案，如直接提问或思维链（chain-of-thought）方法。为方便后续使用，我们发布了一批适用于多种任务的通用且有效的草图，以及一个名为dclib的开源库，用于支撑支持草图功能的解码器运行。
英文摘要：
Many recent prompting strategies for large language models (LLMs) query the model multiple times sequentially -- first to produce intermediate results and then the final answer. However, using these methods, both decoder and model are unaware of potential follow-up prompts, leading to disconnected and undesirably wordy intermediate responses. In this work, we address this issue by proposing prompt sketching, a new prompting paradigm in which an LLM does not only respond by completing a prompt, but by predicting values for multiple variables in a template. This way, sketching grants users more control over the generation process, e.g., by providing a reasoning framework via intermediate instructions, leading to better overall results. The key idea enabling sketching with existing, autoregressive models is to adapt the decoding procedure to also score follow-up instructions during text generation, thus optimizing overall template likelihood in inference. Our experiments show that in a zero-shot setting, prompt sketching outperforms existing, sequential prompting schemes such as direct asking or chain-of-thought on 7 out of 8 LLM benchmarking tasks, including state tracking, arithmetic reasoning, and general question answering. To facilitate future use, we release a number of generic, yet effective sketches applicable to many tasks, and an open source library called dclib, powering our sketch-aware decoders.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
