我们离真正的不可学习性还有多远？
How Far Are We from True Unlearnability?
AI总结：
本文发现现有不可学习样本在多任务数据集上缺乏跨任务不可学习性，从模型优化和损失景观角度解释原因，提出SAL量化参数不可学习性及UD度量数据不可学习性，并对主流方法进行基准测试。
摘要：
高质量数据在大模型时代扮演着不可或缺的角色，但使用未经授权的数据进行模型训练严重损害了数据所有者的利益。为了克服这一威胁，研究者提出了多种不可学习方法，通过破坏数据的训练可用性来生成不可学习样本（UEs）。显然，由于训练目的未知且现有模型具有强大的表示学习能力，这些数据预期对跨多个任务的模型都是不可学习的，即它们不会帮助提升模型性能。然而，出乎意料的是，我们发现在多任务数据集Taskonomy上，UEs在语义分割等任务上仍然表现良好，未能展现出跨任务的不可学习性。这一现象引发我们的疑问：我们离真正实现不可学习样本还有多远？我们尝试从模型优化的角度回答这个问题。为此，我们使用一个简单的模型架构观察干净模型与中毒模型在收敛过程中的差异。随后，从损失景观中我们发现只有部分关键参数优化路径表现出显著差异，这表明损失景观与不可学习性之间存在密切关系。因此，我们利用损失景观来解释UEs失效的根本原因，并基于此解释提出锐度感知可学习性（SAL）来量化参数的不可学习性。此外，我们提出一种不可学习距离（UD），基于干净模型和中毒模型中参数的SAL分布来衡量数据的不可学习性。最后，我们使用所提出的UD对主流不可学习方法进行基准测试，旨在促进社区对现有不可学习方法能力边界的认识。
英文摘要：
High-quality data plays an indispensable role in the era of large models, but the use of unauthorized data for model training greatly damages the interests of data owners. To overcome this threat, several unlearnable methods have been proposed, which generate unlearnable examples (UEs) by compromising the training availability of data. Clearly, due to unknown training purposes and the powerful representation learning capabilities of existing models, these data are expected to be unlearnable for models across multiple tasks, i.e., they will not help improve the model's performance. However, unexpectedly, we find that on the multi-task dataset Taskonomy, UEs still perform well in tasks such as semantic segmentation, failing to exhibit cross-task unlearnability. This phenomenon leads us to question: How far are we from attaining truly unlearnable examples? We attempt to answer this question from the perspective of model optimization. To this end, we observe the difference in the convergence process between clean and poisoned models using a simple model architecture. Subsequently, from the loss landscape we find that only a part of the critical parameter optimization paths show significant differences, implying a close relationship between the loss landscape and unlearnability. Consequently, we employ the loss landscape to explain the underlying reasons for UEs and propose Sharpness-Aware Learnability (SAL) to quantify the unlearnability of parameters based on this explanation. Furthermore, we propose an Unlearnable Distance (UD) to measure the unlearnability of data based on the SAL distribution of parameters in clean and poisoned models. Finally, we conduct benchmark tests on mainstream unlearnable methods using the proposed UD, aiming to promote community awareness of the capability boundaries of existing unlearnable methods.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
