Recent works have shown that generative sequence models (e.g., language models) have a tendency to memorize rare or unique sequences in the training data. Since useful models are often trained on sensitive data, to ensure the privacy of the training data it is critical to identify and mitigate such unintended memorization. Federated Learning (FL) has emerged as a novel framework for large-scale distributed learning tasks. However, it differs in many aspects from the well-studied central learning setting where all the data is stored at the central server. In this paper, we initiate a formal study to understand the effect of different components of canonical FL on unintended memorization in trained models, comparing with the central learning setting. Our results show that several differing components of FL play an important role in reducing unintended memorization. Specifically, we observe that the clustering of data according to users---which happens by design in FL---has a significant effect in reducing such memorization, and using the method of Federated Averaging for training causes a further reduction. We also show that training with a strong user-level differential privacy guarantee results in models that exhibit the least amount of unintended memorization.
Unintended Memorization of Sensitive Information in Fine-Tuned Language Models
TUM University Hospital（慕尼黑工业大学医院）；Technical University of Munich (TUM)（慕尼黑技术大学）；Munich Center for Machine Learning (MCML)（慕尼黑机器学习中心）；Department of Computing, Imperial College London（伦敦帝国学院计算系）
Promise and Peril of Collaborative Code Generation Models: Balancing Effectiveness and Memorization
Centre for Research on Intelligent Software Engineering（智能软件工程研究中心）；School of Computing and Information Systems（计算与信息系统学院）；Singapore Management University（新加坡管理大学）
TriShield: Zero-Utility-Loss Defense Against Privacy Backdoors in Federated Language Model Fine-Tuning via Orthogonal Gradient Projection and Optimizer State Entanglement
ToFU: Transforming How Federated Learning Systems Forget User Data
School of Computer Science（计算机科学学院）；Statistics, Trinity College Dublin, Ireland（统计学，都柏林信任学院，爱尔兰）；School of Computer Science, University College Dublin, Ireland（计算机科学学院，都柏林大学学院，爱尔兰）
Learning Locally, Revising Globally: Global Reviser for Federated Learning with Noisy Labels
College of Computer Science, Sichuan University, Chengdu, China（四川大学计算机学院，中国成都）；Engineering Research Center of Machine Learning（机器学习工程研究中心）；University of Sydney, Sydney, Australia（悉尼大学，澳大利亚悉尼）；Southeast University, Nanjing, China（东南大学，中国南京）
FedCARE: Federated Unlearning with Conflict-Aware Projection and Relearning-Resistant Recovery
School of Cyber Engineering, Xidian University（电子科技大学信息工程学院）；RMIT University（皇家墨尔本理工大学）；Commonwealth Scientific and Industrial Research Organisation (CSIRO)（澳大利亚联邦科学与工业研究组织）；UNSW Sydney（新南威尔士大学悉尼分校）；University of California, Davis（加州大学戴维斯分校）
Training Production Language Models without Memorizing User Data
Google LLC（谷歌有限责任公司）
本文提出了第一个消费者规模的下一个词预测（NWP）模型，该模型使用联邦学习（FL）进行训练，同时利用差分私有联邦平均（DP-FedAvg）技术。已经有关于构建实用FL基础设施的先前工作，包括展示使用这种基础设施在移动设备上训练语言模型的可行性的工作。它也被证明（在公共语料库上的模拟），它是可能的训练NWP模型与用户级差分隐私使用DP-FedAvg算法。然而，在现实世界的生产环境中，在一个异构的手机上，用DP-FedAvg训练生产质量的NWP模型需要解决许多挑战。例如，协调中央服务器必须在每轮开始时跟踪可用的设备，并从它们中随机均匀地采样设备，同时确保\emph{secrecy of the sample}等。与我们所知道的所有先前的以隐私为中心的FL工作不同，我们首次展示了用于在FL中训练生产神经网络的差异隐私机制的部署，以及生产培训基础设施的仪器化，以执行对非故意记忆的端到端经验测量。
University of Maryland, College Park（马里兰大学帕克分校）；Google Research（谷歌研究院）
大型Transformer模型在许多自然语言任务中取得了令人印象深刻的性能。特别地，基于Transformer的语言模型已经被证明在以其大量参数对事实知识进行编码方面具有很强的能力。虽然提高Transformers的记忆和概括能力的任务已经得到了广泛的研究，但如何使transformers忘记特定的旧事实并记住新事实还不太清楚。在本文中，我们提出了一个新的任务\emph{explicitly modifying specific factual knowledge in Transformer models while ensuring the model performance does not degrade on the unmodified facts}。此任务在许多情况下都很有用，例如更新陈旧的知识，保护隐私，以及消除存储在模型中的意外偏见。我们对几种方法进行了基准测试，这些方法在这项任务上提供了自然的基线性能。这导致了Transformer模型的关键组件的发现，这些组件对于知识修改特别有效。这项工作还提供了不同的训练阶段（如预训练和微调）对记忆和知识修改的作用的见解。
Applied Federated Model Personalisation in the Industrial Domain: A Comparative Study
University of Western Macedonia（西马其顿大学）；Kingston University（金斯顿大学）；University of Macedonia（马其顿大学）；Harokopio University of Athens（雅典哈罗科皮奥大学）；Sidroco Holdings Ltd.（锡德罗科控股有限公司）；MetaMind Innovations P.C.（元思维创新私人有限公司）
