迈向检测端到端生成的任务导向对话中的不一致性
Towards Detecting Inconsistencies in End-to-end Generated TODs
- Fondazione Bruno Kessler（布鲁诺·凯斯勒基金会）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
研究端到端生成的任务导向对话中的不一致性检测问题，核心方法是将其概念化为约束满足问题，通过特定管道识别变量和有效解决方案，主要贡献是证明该方法检测不一致性的高精度并提供详细分析。
AI中文摘要：
生成式人工智能正在深刻改变对话系统背后的核心技术，从基于组件的方法转向端到端方法。然而，大语言模型仍可能产生不一致性，这在任务导向对话中是个关键问题，因为系统回复必须严格遵循领域知识库中的信息。一个幻觉（如推荐不存在的餐厅）可能导致严重的任务失败。我们研究一种将任务导向对话概念化为约束满足问题来自动检测不一致性的方法，其中变量代表参考对话领域的对话片段，变量间的约束捕捉诸如轮次连贯性和遵循领域知识等对话属性。我们提出一个管道，先识别目标对话中的变量，然后应用约束满足问题求解器识别有效解决方案。通过将目标对话与有效的变量赋值进行比较，我们能检测不一致性并建议最小更改以确保对话一致性。我们证明了基于约束满足问题的方法检测不一致性的高精度，并对我们的发现进行了详细分析。
英文摘要：
Generative AI is profoundly transforming the core technologies behind conversational systems, shifting from component-based to end-to-end approaches. However, Large Language Models (LLMs) may still generate inconsistencies, a critical issue particularly in Task-Oriented Dialogues (TODs), where system responses must strictly adhere to information from a domain knowledge base (e.g., restaurants in a city). A single hallucination (e.g., suggesting a non-existent restaurant) can lead to severe task failures. We investigate a method for automatically detecting inconsistencies by conceptualizing TODs as a Constraint Satisfaction Problem (CSP), where variables represent dialogue segments referencing the conversational domain, and constraints among variables capture dialogue properties such as turn coherence and adherence to domain knowledge. We propose a pipeline that first identifies variables in a target dialogue and then applies a CSP solver to identify valid solutions. By comparing the target dialogue with valid variable assignments, we can detect inconsistencies and suggest minimal changes to ensure dialogue consistency. We demonstrate the high accuracy of the CSP-based approach in detecting inconsistencies, and provide a detailed analysis of our findings.
