发表机构
Peking University; Singapore Management University; China Telecom Quantum Information Technology Group Co., Ltd; National University of Singapore（北京大学; 新加坡管理大学; 中国电信量子信息科技集团有限公司; 新加坡国立大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
提出zkLLMPoT零知识框架，通过前向评估认证大语言模型训练检查点属性，成本与训练迭代无关，支持高效证明与验证。
AI 中文摘要
当模型权重和训练数据为私有时，审计大语言模型（LLM）训练所声称的结果具有挑战性，而在Transformer规模下以密码学方式证明整个训练过程则代价过高。我们提出了zkLLMPoT，一个零知识框架，通过前向评估而非验证其优化轨迹来认证训练后检查点的审计者定义属性。zkLLMPoT包含两个阶段：1）训练者固定架构并承诺模型权重。然后审计者选择挑战序列，防止训练者针对审计数据修改检查点。2）随后训练者证明承诺模型在这些序列上达到的目标值。这一表述使得认证成本与训练迭代次数无关，同时不泄露模型权重或无需访问私有训练数据。我们基于sumcheck和查找参数来认证Transformer计算，同时支持下一词元损失和特定任务的审计目标。在四个模型家族中，算子级基准测试显示，对于1.1-1.5B参数的模型，证明时间为41-59秒，对于13B模型，所覆盖算子的证明时间为131秒，而在序列长度为512时，验证时间低于半秒。
英文摘要
Auditing the claimed outcomes of large language model (LLM) training is challenging when model weights and training data are private, while cryptographically proving the full training process is prohibitively expensive at Transformer scale. We present zkLLMPoT, a zero-knowledge framework that certifies auditor-defined properties of a trained checkpoint through forward evaluation rather than verification of its optimization trajectory. zkLLMPoT includes 2 phases: 1) The trainer fixes the architecture and the model weights are committed. Then the auditor selects challenge sequences, preventing the trainer from modifying the checkpoint in response to the audit data. 2) Then the trainer proves the objective value attained by the committed model on those sequences. This formulation makes the certification cost independent of the number of training iterations, without revealing model weights or requiring access to private training data. We build on sumcheck- and lookup-based arguments to certify Transformer computations, while supporting next-token loss and task-specific audit objectives. Across four model families, operator-level benchmarks yield proving times of 41-59 seconds for 1.1-1.5B-parameter models and 131 seconds at 13B for the covered operators, with verification below half a second at a sequence length of 512.
CommentsSubmitted to ICLR 2027
