超越连贯性的事实性：评估用于医学文本的 LLM 水印方法
Factuality Beyond Coherence: Evaluating LLM Watermarking Methods for Medical Texts
AI总结：
本文针对医学文本中 LLM 水印可能损害事实性的问题，提出联合评估事实性与连贯性的医学专用流程及事实性加权分数（FWS），并揭示现有水印方法会因熵偏移显著损害医学事实性，呼吁开发领域感知的水印方法。
摘要：
随着大语言模型（LLMs）被适配到医学等敏感领域，其流畅性带来了安全风险，尤其是在来源追溯与问责方面。水印通过嵌入可检测的模式来缓解这些风险，然而其在医学语境中的可靠性尚未得到检验。现有基准主要关注检测质量之间的权衡，而忽视了事实性风险。在医学文本中，水印通常会重新加权低熵 token，这些 token 高度可预测，并且往往承载着关键的医学术语。对这些 token 的偏移可能导致不准确和幻觉，而先前的通用领域基准未能捕捉到这些风险。我们提出了一种以医学为重点的评估工作流程，联合评估事实准确性和连贯性。通过使用 GPT-Judger 并辅以人工验证，我们引入了事实性加权分数（FWS），这是一个复合指标，在连贯性之外优先考虑事实准确性，以指导水印在医学领域的部署。我们的评估表明，当前的水印方法会显著损害医学事实性，熵偏移会降低医学实体的表示质量。这些发现强调了需要具备领域感知能力的水印方法，以保持医学内容的完整性。
英文摘要：
As large language models (LLMs) are adapted to sensitive domains such as medicine, their fluency raises safety risks, particularly regarding provenance and accountability. Watermarking embeds detectable patterns to mitigate these risks, yet its reliability in medical contexts remains untested. Existing benchmarks focus on detection-quality tradeoffs and overlook factual risks. In medical text, watermarking often reweights low-entropy tokens, which are highly predictable and often carry critical medical terminology. Shifting these tokens can cause inaccuracy and hallucinations, risks that prior general-domain benchmarks fail to capture. We propose a medical-focused evaluation workflow that jointly assesses factual accuracy and coherence. Using GPT-Judger and further human validation, we introduce the Factuality-Weighted Score (FWS), a composite metric prioritizing factual accuracy beyond coherence to guide watermarking deployment in medical domains. Our evaluation shows current watermarking methods substantially compromise medical factuality, with entropy shifts degrading medical entity representation. These findings underscore the need for domain-aware watermarking approaches that preserve the integrity of medical content.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
