ConstitutionMaker:通过将反馈转化为原则来交互式地评判大型语言模型
ConstitutionMaker: Interactively Critiquing Large Language Models by Converting Feedback into Principles
AI总结:
研究提出交互式工具 ConstitutionMaker,通过将用户自然语言反馈自动转化为指导模型行为的原则(宪法)并插入提示中,帮助用户更高效、低负担地优化大型语言模型聊天机器人的输出。
摘要:
大型语言模型(LLM)提示是一种很有前景的新方法,用户可以通过它创建和定制自己的聊天机器人。然而,当前引导聊天机器人输出的方法,如提示工程和微调,不支持用户将他们对模型输出的自然反馈转化为提示或模型的改变。在这项工作中,我们探索了如何通过帮助用户将他们的反馈转化为一组决定模型行为的原则(即宪法),来使用户能够通过反馈交互式地优化模型输出。通过一项形成性研究,我们(1)发现用户需要支持将他们的反馈转化为聊天机器人的原则,并且(2)对用户期望的不同原则类型进行了分类。受这些发现的启发,我们开发了 ConstitutionMaker,这是一个将用户反馈转化为原则以引导基于 LLM 的聊天机器人的交互式工具。使用 ConstitutionMaker,用户可以用自然语言提供正面或负面反馈,选择自动生成的反馈,或重写聊天机器人的回复;每种反馈模式都会自动生成一条原则,并插入到聊天机器人的提示中。在一项有 14 名参与者参与的用户研究中,我们将 ConstitutionMaker 与一个让用户自己编写原则的消融版本进行了比较。使用 ConstitutionMaker 时,参与者认为他们的原则能更好地引导聊天机器人,他们能更容易地将反馈转化为原则,并且能更高效地编写原则,心理负担也更小。ConstitutionMaker 帮助用户找到了改进聊天机器人的方法,将他们对模型的直观回应转化为反馈,并将这种反馈转化为具体明确的原则。这些发现共同为未来支持交互式评判 LLM 输出的工具提供了参考。
英文摘要:
Large language model (LLM) prompting is a promising new approach for users to create and customize their own chatbots. However, current methods for steering a chatbot's outputs, such as prompt engineering and fine-tuning, do not support users in converting their natural feedback on the model's outputs to changes in the prompt or model. In this work, we explore how to enable users to interactively refine model outputs through their feedback, by helping them convert their feedback into a set of principles (i.e. a constitution) that dictate the model's behavior. From a formative study, we (1) found that users needed support converting their feedback into principles for the chatbot and (2) classified the different principle types desired by users. Inspired by these findings, we developed ConstitutionMaker, an interactive tool for converting user feedback into principles, to steer LLM-based chatbots. With ConstitutionMaker, users can provide either positive or negative feedback in natural language, select auto-generated feedback, or rewrite the chatbot's response; each mode of feedback automatically generates a principle that is inserted into the chatbot's prompt. In a user study with 14 participants, we compare ConstitutionMaker to an ablated version, where users write their own principles. With ConstitutionMaker, participants felt that their principles could better guide the chatbot, that they could more easily convert their feedback into principles, and that they could write principles more efficiently, with less mental demand. ConstitutionMaker helped users identify ways to improve the chatbot, formulate their intuitive responses to the model into feedback, and convert this feedback into specific and clear principles. Together, these findings inform future tools that support the interactive critiquing of LLM outputs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。