IP-Dialog：利用合成数据评估对话系统中的隐式个性化
IP-Dialog: Evaluating Implicit Personalization in Dialogue Systems with Synthetic Data
AI总结：
本文提出IP-Dialog基准及合成数据生成方法，覆盖10项任务和12种用户属性，并通过四指标评估框架与因果图系统评估对话系统的隐式个性化推理能力。
摘要：
在现代对话系统中，从对话中隐式推断用户背景并利用这些信息提供个性化辅助的能力至关重要。然而，高质量数据的稀缺仍然是评估和提升这一能力的根本挑战。传统数据集构建方法劳动密集、资源需求高，并引发隐私问题。为解决这些问题，我们提出了一种自动合成数据生成的新方法，并引入了隐式个性化对话（IP-Dialog）基准及训练数据集，涵盖10项任务和12种用户属性类型。此外，我们开发了一个包含四个指标的系统化评估框架，用于评估属性感知和推理能力。我们进一步提出五个因果图，以阐明模型在隐式个性化过程中的推理路径。大量实验得出了富有洞察力的观察结果，并证明了我们数据集的可靠性。
英文摘要：
In modern dialogue systems, the ability to implicitly infer user backgrounds from conversations and leverage this information for personalized assistance is crucial. However, the scarcity of high-quality data remains a fundamental challenge to evaluating and improving this capability. Traditional dataset construction methods are labor-intensive, resource-demanding, and raise privacy concerns. To address these issues, we propose a novel approach for automatic synthetic data generation and introduce the Implicit Personalized Dialogue (IP-Dialog) benchmark along with a training dataset, covering 10 tasks and 12 user attribute types. Additionally, we develop a systematic evaluation framework with four metrics to assess both attribute awareness and reasoning capabilities. We further propose five causal graphs to elucidate models' reasoning pathways during implicit personalization. Extensive experiments yield insightful observations and prove the reliability of our dataset.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
