Watermarking is a principled approach for tracing the provenance of large language model (LLM) outputs, but its deployment in practice is hindered by inference inefficiency. Speculative sampling accelerates inference, with efficiency improving as the acceptance rate between draft and target models increases. Yet recent work reveals a fundamental trade-off: higher watermark strength reduces acceptance, preventing their simultaneous achievement. We revisit this trade-off and show it is not absolute. We introduce a quantitative measure of watermark strength that governs statistical detectability and is maximized when tokens are deterministic functions of pseudorandom numbers. Using this measure, we fully characterize the trade-off as a constrained optimization problem and derive explicit Pareto curves for two existing watermarking schemes. Finally, we introduce a principled mechanism that injects pseudorandomness into draft-token acceptance, ensuring maximal watermark strength while maintaining speculative sampling efficiency. Experiments further show that this approach improves detectability without sacrificing efficiency. Our findings uncover a principle that unites speculative sampling and watermarking, paving the way for their efficient and practical deployment.
Towards Better Statistical Understanding of Watermarking LLMs
Imperial College Business School, Imperial College London（帝国理工学院商学院，帝国理工学院）；The University of Sydney Business School, The University of Sydney（悉尼大学商学院，悉尼大学）；Grado Department of Industrial and Systems Engineering, Virginia Tech（弗吉尼亚理工大学工业与系统工程系）
With the increasing application value of machine learning, the intellectual property (IP) rights of deep neural networks (DNN) are getting more and more attention. With our analysis, most of the existing DNN watermarking methods can resist fine-tuning and pruning attack, but distillation attack. To address these problem, we propose a new DNN watermarking framework, Unified Soft-label Perturbation (USP), having a detector paired with the model to be watermarked, and Customized Soft-label Perturbation (CSP), embedding watermark via adding perturbation into the model output probability distributi…
FreqMark: Invisible Image Watermarking via Frequency Based Optimization in Latent Space
University of California, Santa Cruz（加州大学圣克鲁兹分校）；The University of Hong Kong（香港大学）；University of Science and Technology of China（中国科学技术大学）；IEG, Tencent（腾讯IEG）；Fudan University（复旦大学）
