基于声明式Genie工作表的可控可靠知识密集型任务导向对话智能体
Controllable and Reliable Knowledge-Intensive Task-Oriented Conversational Agents with Declarative Genie Worksheets
AI总结：
针对大语言模型对话智能体的幻觉、条件逻辑遵循差等问题，提出可编程框架Genie，通过声明式工作表和运行时管控实现可控可靠的对话，在复杂数据集及三项真实任务中性能显著提升。
摘要：
大语言模型能够在多种场景下开展类人对话，响应用户的任务与知识请求。然而，现有基于大语言模型构建的对话智能体往往存在幻觉问题，难以遵循含条件逻辑的指令，也难以整合多来源知识，这些缺陷削弱了智能体的有效性，使其不适合部署。\n为应对这些挑战，我们提出Genie——一款用于构建知识密集型任务导向对话智能体的可编程框架。Genie能够处理复杂交互、回答复杂查询。与大语言模型不同，它通过先进的对话状态管理生成可靠、有依据的回复，并通过其声明式规范Genie工作表支持可控的智能体策略。这一特性由算法运行时系统实现：该系统执行开发者提供的策略，将大语言模型的作用限定为两点：一是利用精简的对话历史解析用户输入，二是根据提供的上下文生成回复。\n基于Genie构建的智能体在复杂逻辑对话数据集上的表现优于SOTA方法。我们针对三个真实应用开展了有62名参与者的用户研究：基于Yelp的餐厅预订，以及面向大学生的工单提交和课程注册。搭载GPT-4 Turbo的Genie智能体表现优于使用函数调用的GPT-4 Turbo智能体，在三项真实任务中将目标完成率从21.8%提升至82.8%。
英文摘要：
Large Language Models can carry out human-like conversations in diverse settings, responding to user requests for tasks and knowledge. However, existing conversational agents implemented with LLMs often struggle with hallucination, following instructions with conditional logic, and integrating knowledge from different sources. These shortcomings compromise the agents' effectiveness, rendering them unsuitable for deployment. To address these challenges, we introduce Genie, a programmable framework for creating knowledge-intensive task-oriented conversational agents. Genie can handle involved interactions and answer complex queries. Unlike LLMs, it delivers reliable, grounded responses through advanced dialogue state management and supports controllable agent policies via its declarative specification -- Genie Worksheet. This is achieved through an algorithmic runtime system that implements the developer-supplied policy, limiting LLMs to (1) parse user input using a succinct conversational history, and (2) generate responses according to supplied context. Agents built with Genie outperform SOTA methods on complex logic dialogue datasets. We conducted a user study with 62 participants on three real-life applications: restaurant reservations with Yelp, as well as ticket submission and course enrollment for university students. Genie agents with GPT-4 Turbo outperformed the GPT-4 Turbo agents with function calling, improving goal completion rates from 21.8% to 82.8% across three real-world tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
