SAFE：基于分片图的机器遗忘
SAFE: Machine Unlearning With Shard Graphs
AI总结：
本研究提出SAFE机器遗忘方法，通过引入分片图结合轻量级适配器，在分片规模比现有最优方法小一个数量级的情况下，兼顾低遗忘成本与高准确率，且能彻底清除训练样本的残余影响。
摘要：
我们提出了协同感知遗忘集成（Synergy Aware Forgetting Ensemble，SAFE）方法，该方法可在多样化数据集上适配大模型，同时将从训练好的模型中移除训练样本影响的预期成本降至最低。这一过程也被称为选择性遗忘或机器遗忘，通常的实现方式是将数据集划分为多个分片，在每个分片上训练完全独立的模型，再对所得模型进行集成。增加分片数量会降低预期遗忘成本，但同时会提升推理成本，且由于独立模型训练过程中丢失了样本间的协同信息，模型的最终准确率也会下降。SAFE并未将每个分片视为独立个体，而是引入了分片图的概念，允许在训练过程中融入来自其他分片的有限信息，以预期遗忘成本的小幅上升换取准确率的显著提升，同时仍能在遗忘后彻底清除残余影响。SAFE采用轻量级适配器系统，可在复用大部分计算量的情况下完成训练。我们在细粒度计算机视觉数据集上的实验验证表明，这使得SAFE能够在比当前最优方法小一个数量级的分片上训练（从而降低遗忘成本），同时还能保持高准确率。
英文摘要：
We present Synergy Aware Forgetting Ensemble (SAFE), a method to adapt large models on a diverse collection of data while minimizing the expected cost to remove the influence of training samples from the trained model. This process, also known as selective forgetting or unlearning, is often conducted by partitioning a dataset into shards, training fully independent models on each, then ensembling the resulting models. Increasing the number of shards reduces the expected cost to forget but at the same time it increases inference cost and reduces the final accuracy of the model since synergistic information between samples is lost during the independent model training. Rather than treating each shard as independent, SAFE introduces the notion of a shard graph, which allows incorporating limited information from other shards during training, trading off a modest increase in expected forgetting cost with a significant increase in accuracy, all while still attaining complete removal of residual influence after forgetting. SAFE uses a lightweight system of adapters which can be trained while reusing most of the computations. This allows SAFE to be trained on shards an order-of-magnitude smaller than current state-of-the-art methods (thus reducing the forgetting costs) while also maintaining high accuracy, as we demonstrate empirically on fine-grained computer vision datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
