会话回归测试：一种为预训练语言模型原型化可推广提示策略的设计技术
Conversation Regression Testing: A Design Technique for Prototyping Generalizable Prompt Strategies for Pre-trained Language Models
摘要：
预训练的语言模型（LLM），如GPT-3，可以进行流畅的、多回合的对话，使它们成为聊天机器人设计的有吸引力的材料。此外，设计人员可以通过将文本提示（指令和所需交互的示例）前置到其输入来改进LLM聊天机器人的话语。然而，基于即时策略的改进可能是脆弱的;设计师面临着系统地理解即时策略如何影响用户之间后续对话的挑战。为了应对这一挑战，我们引入了会话回归测试的概念。基于与基线聊天机器人的示例对话，对话回归测试跟踪对话错误如何持续存在或通过应用不同的提示策略来解决。我们将这种技术体现在一个交互式设计工具BotDesigner中，该工具可以让设计人员在多个对话中识别原型错误;使用图形可视化显示对话的共同线程;并突出显示机器人设计迭代中提示更改的效果。试点评估证明了回归测试的概念和聊天机器人设计师的BotDesigner功能的有用性。
英文摘要：
Pre-trained language models (LLMs) such as GPT-3 can carry fluent, multi-turn conversations out-of-the-box, making them attractive materials for chatbot design. Further, designers can improve LLM chatbot utterances by prepending textual prompts -- instructions and examples of desired interactions -- to its inputs. However, prompt-based improvements can be brittle; designers face challenges systematically understanding how a prompt strategy might impact the unfolding of subsequent conversations across users. To address this challenge, we introduce the concept of Conversation Regression Testing. Based on sample conversations with a baseline chatbot, Conversation Regression Testing tracks how conversational errors persist or are resolved by applying different prompt strategies. We embody this technique in an interactive design tool, BotDesigner, that lets designers identify archetypal errors across multiple conversations; shows common threads of conversation using a graph visualization; and highlights the effects of prompt changes across bot design iterations. A pilot evaluation demonstrates the usefulness of both the concept of regression testing and the functionalities of BotDesigner for chatbot designers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
