PrivMedChat: 医疗对话系统的端到端差分隐私RLHF
PrivMedChat: End-to-End Differentially Private RLHF for Medical Dialogue Systems
AI总结：
PrivMedChat通过端到端差分隐私RLHF方法，在医疗对话系统中实现隐私保护与对齐，采用DP-SGD和DP-aware策略优化，避免临床标注成本，提供正式隐私保证。
摘要：
大型语言模型越来越多地用于面向患者的医疗协助和临床决策支持，但将其适应到临床对话往往需要来自医生-患者对话的监督信息，这些信息可能包含敏感信息。传统监督微调和人类反馈强化学习（RLHF）可能会放大记忆，导致成员推断和披露罕见训练集细节。我们提出了PrivMedChat（隐私医疗聊天），一种用于医疗对话系统的端到端差分隐私RLHF（DP-RLHF）框架。我们的方法在每个训练阶段都强制实施差分隐私，结合DP-SGD进行监督微调和偏好对的奖励模型学习，以及DP-aware策略优化进行对齐。为了避免昂贵的临床标注，我们引入了一种无需标注的偏好构建策略，将医生回应与过滤后的非专家生成内容配对。我们在医疗对话任务上评估PrivMedChat，并在一致的隐私计费下评估其效用、安全性和隐私，从而提供了一条实际路径来对齐医疗聊天机器人，同时提供正式的隐私保证。我们开源了我们的代码在https://github.com/sudip-bhujel/privmedchat。
英文摘要：
Large language models are increasingly used for patient-facing medical assistance and clinical decision support, but adapting them to clinical dialogue often requires supervision derived from doctor-patient conversations that may contain sensitive information. Conventional supervised fine-tuning and reinforcement learning from human feedback (RLHF) can amplify memorization, enabling membership inference and disclosure of rare training-set details. We present PrivMedChat (Private Medical Chat), an end-to-end framework for differentially private RLHF (DP-RLHF) for medical dialogue systems. Our approach enforces differential privacy at each training stage that accesses dialogue-derived supervision, combining DP-SGD for supervised fine-tuning and reward model learning from preference pairs, and DP-aware policy optimization for alignment. To avoid costly clinician labeling, we introduce an annotation-free preference construction strategy that pairs physician responses with filtered non-expert generations. We evaluate PrivMedChat across medical dialogue tasks and assess utility, safety, and privacy under consistent privacy accounting, thereby providing a practical pathway to align medical chatbots while offering formal privacy guarantees. We open-source our code at https://github.com/sudip-bhujel/privmedchat.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
