参数还是隐私：过度参数化与成员推理之间的可证明权衡
Parameters or Privacy: A Provable Tradeoff Between Overparameterization and Membership Inference
摘要：
现代机器学习中的一个令人惊讶的现象是，高度过参数化的模型即使经过训练以记住训练数据（训练数据上的零错误），也能够很好地泛化（测试数据上的小错误）。这导致了对越来越多的过度参数化模型的军备竞赛（参见，深度学习）。在本文中，我们研究了一个未充分探索的隐藏成本overparameterization：事实上，overparameterized模型可能更容易受到隐私攻击，特别是成员推理攻击，预测（潜在的敏感）的例子用于训练模型。我们显着地扩展了相对较少的经验结果在这个问题上的理论证明了一个过参数化的线性回归模型在高斯数据设置的成员推断脆弱性增加的参数的数量。此外，一系列实证研究表明，更复杂的非线性模型也表现出同样的行为。最后，我们将我们的分析扩展到岭正则化线性回归，并在高斯数据设置中显示，增加正则化也增加了过度参数化制度中的成员推断脆弱性。
英文摘要：
A surprising phenomenon in modern machine learning is the ability of a highly overparameterized model to generalize well (small error on the test data) even when it is trained to memorize the training data (zero error on the training data). This has led to an arms race towards increasingly overparameterized models (c.f., deep learning). In this paper, we study an underexplored hidden cost of overparameterization: the fact that overparameterized models may be more vulnerable to privacy attacks, in particular the membership inference attack that predicts the (potentially sensitive) examples used to train a model. We significantly extend the relatively few empirical results on this problem by theoretically proving for an overparameterized linear regression model in the Gaussian data setting that membership inference vulnerability increases with the number of parameters. Moreover, a range of empirical studies indicates that more complex, nonlinear models exhibit the same behavior. Finally, we extend our analysis towards ridge-regularized linear regression and show in the Gaussian data setting that increased regularization also increases membership inference vulnerability in the overparameterized regime.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
