arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2603.01550cs.CLcs.AI
收藏

从基于大语言模型的任务机器人中提取训练对话数据

Extracting Training Dialogue Data from Large Language Model based Task Bots

MOE Key Laboratory for Intelligent Networks and Network Security, Xi’an Jiaotong University（信息网络与网络安全国家重点实验室，西安交通大学）

Shuo Zhang, Junzhou Zhao, Junji Hou, Pinghui Wang, Chenxu Wang, Jing Tao

备注：
Accepted for publication in IEEE Transactions on Information Forensics and Security (TIFS). \c{opyright} 2026 IEEE
阅读 PDF
arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出了一种针对基于大语言模型的任务机器人数据提取攻击方法，通过系统性研究揭示LLM记忆在TODS中的继承机制，并有效提取训练对话数据。

摘要：

大型语言模型（LLMs）已被广泛采用，以通过建模复杂的语言模式和提供上下文合适的响应来增强任务导向对话系统（TODS）。然而，这种整合引入了显著的隐私风险，因为LLMs作为软知识库，将大量的训练数据压缩成丰富的知识表示，可能会无意中记住包含电话号码等可识别信息以及完整对话层面事件如完整旅行日程的训练对话数据。尽管这一隐私问题至关重要，但LLM记忆在开发任务机器人中的继承方式仍不清楚。在本工作中，我们通过系统性的定量研究来填补这一空白，该研究包括评估现有的训练数据提取攻击、分析关键特征，这些特征使现有方法无效，并提出针对基于LLM的TODS的新型攻击技术，以增强响应采样和成员推断。实验结果证明了我们提出的数据提取攻击的有效性。我们的方法可以提取数千个训练标签的对话状态，最佳情况下精度超过70%。此外，我们还深入分析了基于LLM的TODS中的训练数据记忆，通过识别和量化关键影响因素，并讨论了有针对性的缓解策略。

英文摘要：

Large Language Models (LLMs) have been widely adopted to enhance Task-Oriented Dialogue Systems (TODS) by modeling complex language patterns and delivering contextually appropriate responses. However, this integration introduces significant privacy risks, as LLMs, functioning as soft knowledge bases that compress extensive training data into rich knowledge representations, can inadvertently memorize training dialogue data containing not only identifiable information such as phone numbers but also entire dialogue-level events like complete travel schedules. Despite the critical nature of this privacy concern, how LLM memorization is inherited in developing task bots remains unexplored. In this work, we address this gap through a systematic quantitative study that involves evaluating existing training data extraction attacks, analyzing key characteristics of task-oriented dialogue modeling that render existing methods ineffective, and proposing novel attack techniques tailored for LLM-based TODS that enhance both response sampling and membership inference. Experimental results demonstrate the effectiveness of our proposed data extraction attack. Our method can extract thousands of training labels of dialogue states with best-case precision exceeding 70%. Furthermore, we provide an in-depth analysis of training data memorization in LLM-based TODS by identifying and quantifying key influencing factors and discussing targeted mitigation strategies.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。

↑