Delta注意力中的精确记录遗漏：传输准则、其代价与重放证书
Exact Record Omission in Delta Attention: A Transport Criterion, Its Cost, and a Replay Certificate
- Vy Labs, Inc.（Vy实验室公司）
机构由 AI 辅助整理，请以论文原文为准。
AI总结：
本研究证明传输收据仅在净效应抵消时实现精确记录遗漏，实测48B Kimi Linear等模型不满足，而检查点重放可精确匹配未存储状态。
AI中文摘要：
当用户要求助手遗忘一条记录时，检验标准是记忆是否现在与若该记录从未被存储时所应保持的状态相匹配。独立编码的行可以直接移除；而循环记忆则将记录折叠进一个不断演化的状态中。一种希望是持有收据：保存记录到达时所产生的差异，将其携带至后续更新中，并予以减去，这样无论对话持续多长，删除成本都只需一次固定大小的编辑。我们证明，传输的收据能实现精确遗漏，当且仅当该记录在后续更新中引发的改变在净效应上相互抵消；并且我们在发布的48B Kimi Linear混合模型上测量了这些改变是否如此。结果并非如此：在再经过4096个词元后，该记录仍留下约状态范数4.5%的印记，而所有测试的收据类别均无法去除；重算一半后缀所弥补的差距不足一半，且收据所需的每词元日志在88个词元后成本便超过完整检查点。相同的写入规则分类在Mamba-2、Falcon-H1和RWKV-7上成立，预测在运行前已记录。从记录前的检查点恢复并重放幸存的后续序列，在我们检查的每个数组上均与从未存储的状态精确匹配。在混合后缀扫描中，掩蔽该记录的注意力行使采样恢复接近从未存储的基线水平，尽管循环印记仍然存在，而重建参考的审计者仍能检测到它。在评估的方法中，检查点重放实现了精确遗漏，其工作量与重放的后缀长度成正比。
英文摘要：
An assistant can stop repeating a deleted statement while its recurrent memory still carries that statement's influence. We examine this distinction by saving the state difference immediately after a record, carrying this saved difference, or receipt, through later updates, and comparing the corrected state with the state built from the same conversation with the record omitted. Unrolling the recurrence gives an exact criterion: transport reaches this never-stored state if and only if the additional differences created by later updates cancel after transport. We evaluate eleven conditions using 40 prospectively selected synthetic Kimi Linear contexts, with 40 separate calibration contexts and paired uncertainty estimates computed across complete records. Attention masking removed all 40 exact greedy target answers, yet a fixed three-query candidate-scoring attack with known candidate values and log-probability access attained AUC 0.740 (95% interval 0.695--0.809); the measured false-positive rate was 7.5%. Prompt-only forgetting still returned 33 of the 40 targets. Checkpoint replay matched the complete declared active state and audit logits in all 80 contexts, while preserving all 120 retained answers in the evaluation cohort. An independent Qwen cohort and matched original-bf16/8-bit Kimi controls confirmed native transport mismatch beyond the numerical error measured in matched controls. Replay also supported record replacement and successive deletions with a new record added between them. These results provide a practical audit of both the requested memory change and the assistant's remaining behavior, with replay work determined by the surviving suffix.
