规范化稳定列表回放：面向语言模型嵌入的私有联邦持续学习
Canonicalized Stable-List Replay for Private Federated Continual Learning over Language-Model Embeddings
AI总结：
针对联邦持续学习中差分隐私下回放列表无序的问题，提出规范化稳定列表回放（CSLR）方法，利用公共锚句的签名对齐客户端分布，在多个基准上提升性能。
摘要：
联邦持续学习（FCL）允许分布式客户端在不共享原始文本的情况下，将语言模型头部适应不断演变的NLP任务。在用户级差分隐私（DP）下，基于回放的持续学习面临一个结构性障碍：客户端只能发布候选回放摘要的小型噪声列表，且这些列表在客户端之间是无序的。我们引入了规范化稳定列表回放（CSLR），其中客户端在共享的句子嵌入空间上私有地生成候选回放分布，服务器使用公共锚句诱导的签名对齐它们。锚点提供聚合的可识别性，而不是额外的回放数据。我们证明，在可观测的锚签名间隔下，$O(\log(N/η)/p)$个锚点以至少$1-η$的概率区分$N$个候选列表元素，并给出了无序标签预言机模型的范围性无锚不可识别性结果。在持续分类、NER和对话基准的五个随机种子上，CSLR在报告的回放发布预算下，在$\eps=4$时，最终平均任务指标比最强的非CSLR DP基线提高了3.9-5.6个点，同时也优于匈牙利匹配和最优传输匹配。形式化隐私保证涵盖回放发布；端到端私有训练还需要与用于任务头更新的私有优化器组合。
英文摘要：
Federated continual learning (FCL) lets distributed clients adapt language-model heads to evolving NLP tasks without sharing raw text. Under user-level differential privacy (DP), replay-based continual learning faces a structural obstacle: clients can release only small noisy lists of candidate replay summaries, and those lists are unordered across clients. We introduce Canonicalized Stable-List Replay (CSLR), where clients privately produce candidate replay distributions over a shared sentence-embedding space and the server aligns them using signatures induced by public anchor sentences. The anchors provide identifiability for aggregation rather than additional replay data. We prove that, under an observable anchor-signature margin, $O(\log(N/η)/p)$ anchors distinguish $N$ candidate list elements with probability at least $1-η$, and we give a scoped anchorless non-identifiability result for unordered-label oracle models. Across five seeds on continual classification, NER, and dialogue benchmarks, CSLR improves the final average task metric by 3.9--5.6 points over the strongest non-CSLR DP baseline at $\eps=4$ under the reported replay-release budget, while also outperforming Hungarian and optimal-transport matchers. The formal privacy guarantee covers replay release; end-to-end private training additionally requires composition with a private optimizer for task-head updates.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
