ACING：面向黑盒大语言模型指令学习的演员-评论家方法
ACING: Actor-Critic for Instruction Learning in Black-Box LLMs
AI总结：
针对黑盒大语言模型指令优化难题，提出基于演员-评论家强化学习的ACING框架，将其建模为无状态连续动作问题，仅用黑盒反馈探索指令空间，在多类任务上显著优于人工提示和现有自动基线。
摘要：
大语言模型（LLM）解决任务的效果很大程度上取决于指令的质量，而这些指令通常需要大量人力来编写，这凸显了自动化指令优化的必要性。然而，在处理无法获取模型参数和梯度的黑盒LLM时，指令优化尤其具有挑战性。我们提出了ACING，这是一种演员-评论家强化学习框架，将指令优化建模为无状态的连续动作问题，仅利用黑盒反馈就能在无限指令空间中进行探索。在涵盖指令归纳、摘要生成和思维链推理的33项任务中，ACING自动发现的提示在76%的指令归纳任务上优于人工编写的提示，相比最优自动基线最高提升33分，中位提升达10分。大量消融实验验证了其鲁棒性和高效性。ACING的实现可在https://github.com/salmakh1/ACING获取。
英文摘要：
The effectiveness of Large Language Models (LLMs) in solving tasks depends significantly on the quality of their instructions, which often require substantial human effort to craft. This underscores the need for automated instruction optimization. However, optimizing instructions is particularly challenging when working with black-box LLMs, where model parameters and gradients are inaccessible. We introduce ACING, an actor-critic reinforcement learning framework that formulates instruction optimization as a stateless, continuous-action problem, enabling exploration of infinite instruction spaces using only black-box feedback. ACING automatically discovers prompts that outperform human-written prompts in 76% of instruction-induction tasks, with gains of up to 33 points and a 10-point median improvement over the best automatic baseline in 33 tasks spanning instruction-induction, summarization, and chain-of-thought reasoning. Extensive ablations highlight its robustness and efficiency. An implementation of ACING is available at https://github.com/salmakh1/ACING.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
