通过三智能体生成流水线实现个性化生成式摘要
Personalized Abstractive Summarization by Tri-agent Generation Pipeline
AI总结：
针对大模型输出难以匹配隐式用户偏好的问题，提出由生成器、指导器和编辑器构成的三智能体流水线，并用编辑器引导的强化学习训练小型指导器，显著提升生成式摘要的个性化效果。
摘要：
尽管 ChatGPT 等大型语言模型具备出色的生成能力，但使其输出契合用户的隐式偏好仍是一项挑战。本文提出一种由生成器、指导器和编辑器组成的三智能体生成流水线，以增强输出个性化。生成器生成初始输出，指导器根据用户偏好自动生成编辑指令，编辑器则优化输出，使其与这些偏好保持一致。仅用于推理的大型语言模型（ChatGPT）同时充当生成器和编辑器，另由一个较小模型担任指导器以引导输出生成。我们采用编辑器引导的强化学习训练指导器，利用来自大规模编辑器模型的反馈优化指令生成。在两个生成式摘要数据集上的实验结果表明，本方法能够有效生成更符合用户期望的输出。代码位于 https://github.com/Wendy-Xiao/chatgpt_editing_summ 。
英文摘要：
Tailoring outputs from large language models, like ChatGPT, to implicit user preferences remains a challenge despite their impressive generative capabilities. In this paper, we propose a tri-agent generation pipeline comprising a generator, an instructor, and an editor to enhance output personalization. The generator produces an initial output, the instructor automatically generates editing instructions based on user preferences, and the editor refines the output to align with those preferences. The inference-only large language model (ChatGPT) serves as both the generator and editor, with a smaller model acting as the instructor to guide output generation. We train the instructor using editor-steered reinforcement learning, leveraging feedback from a large-scale editor model to optimize instruction generation. Experimental results on two abstractive summarization datasets demonstrate the effectiveness of our approach in generating outputs that better meet user expectations. Code is available at \url{https://github.com/Wendy-Xiao/chatgpt_editing_summ}
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
