在自然语言长期指令语境下解读用户请求
Interpreting User Requests in the Context of Natural Language Standing Instructions
AI总结：
针对自然语言界面用户需重复偏好的问题，提出将长期指令作为对话建模额外语境的方法，构建覆盖17领域的NLSI数据集，实验显示API预测最高精确匹配率44.7%，凸显相关挑战。
摘要：
通常由大语言模型（LLM）驱动的自然语言界面用户，每次提出类似请求时往往都要重复自己的偏好。我们提出了一种基于LLM的对话建模方法，将持久化的用户约束与偏好——统称为长期指令（standing instructions）——作为此类界面的额外语境。例如，当用户说“我饿了”时，此前表达过的对波斯菜的偏好可被自动添加到LLM提示中，从而影响相关餐厅的搜索。我们构建了NLSI，这是一个包含超2400段对话、覆盖17个领域的语言转程序数据集，每段对话都配对了一个用户档案（一组用户特定的长期指令）及对应的结构化表示（API调用）。NLSI中的一项核心挑战是识别长期指令的哪个子集适用于给定对话。NLSI涵盖多样的现象，从简单偏好到相互依赖的指令，比如每当用户预订活动门票时触发酒店搜索。我们使用大语言模型提示工程及多种检索方法在NLSI上开展实验，在API预测上的最高精确匹配率达到44.7%。我们的结果展现了识别相关长期指令并将其解读为API调用的挑战性。
英文摘要：
Users of natural language interfaces, generally powered by Large Language Models (LLMs),often must repeat their preferences each time they make a similar request. We describe an approach to LLM-based dialogue modeling in which persistent user constraints and preferences -- collectively termed standing instructions -- as additional context for such interfaces. For example, when a user states "I'm hungry", a previously expressed preference for Persian food can be automatically added to the LLM prompt, influencing the search for relevant restaurants. We develop NLSI, a language-to-program dataset consisting of over 2.4K dialogues spanning 17 domains, where each dialogue is paired with a user profile (a set of users specific standing instructions) and corresponding structured representations (API calls). A key challenge in NLSI is to identify which subset of the standing instructions is applicable to a given dialogue. NLSI contains diverse phenomena, from simple preferences to interdependent instructions such as triggering a hotel search whenever the user is booking tickets to an event. We conduct experiments on NLSI using prompting with large language models and various retrieval approaches, achieving a maximum of 44.7% exact match on API prediction. Our results demonstrate the challenges in identifying the relevant standing instructions and their interpretation into API calls.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
