脆弱的巨人:理解模型对子群体攻击的易感性
Fragile Giants: Understanding the Susceptibility of Models to Subpopulation Attacks
AI总结:
本文研究了模型复杂度对子群体投毒攻击易感性的影响,提出理论框架并通过大规模实验证明参数更多的模型更易受此类攻击,强调需开发针对子群体漏洞的防御措施。
摘要:
随着机器学习模型变得日益复杂,对其鲁棒性和可信度的担忧变得更加紧迫。这些模型的一个关键漏洞是数据投毒攻击,攻击者故意篡改训练数据以降低模型性能。此类攻击中一种尤为隐蔽的形式是子群体投毒,它针对数据集中的特定子群体,同时使整体性能基本保持不变。这些攻击在子群体内泛化的能力在现实环境中构成了重大风险,因为它们可能被利用来损害数据集中被边缘化或代表性不足的群体。在本研究中,我们探讨了模型复杂度如何影响对子群体投毒攻击的易感性。我们引入了一个理论框架,解释了过参数化模型如何由于其大容量而无意中记忆并错误分类目标子群体。为了验证我们的理论,我们在大规模图像和文本数据集上使用流行的模型架构进行了广泛的实验。我们的结果显示了一个清晰的趋势:具有更多参数的模型显著更容易受到子群体投毒的影响。此外,我们发现针对较小的、人类可解释的子群体的攻击往往被这些模型忽略。这些结果凸显了开发专门应对子群体漏洞的防御措施的需求。
英文摘要:
As machine learning models become increasingly complex, concerns about their robustness and trustworthiness have become more pressing. A critical vulnerability of these models is data poisoning attacks, where adversaries deliberately alter training data to degrade model performance. One particularly stealthy form of these attacks is subpopulation poisoning, which targets distinct subgroups within a dataset while leaving overall performance largely intact. The ability of these attacks to generalize within subpopulations poses a significant risk in real-world settings, as they can be exploited to harm marginalized or underrepresented groups within the dataset. In this work, we investigate how model complexity influences susceptibility to subpopulation poisoning attacks. We introduce a theoretical framework that explains how overparameterized models, due to their large capacity, can inadvertently memorize and misclassify targeted subpopulations. To validate our theory, we conduct extensive experiments on large-scale image and text datasets using popular model architectures. Our results show a clear trend: models with more parameters are significantly more vulnerable to subpopulation poisoning. Moreover, we find that attacks on smaller, human-interpretable subgroups often go undetected by these models. These results highlight the need to develop defenses that specifically address subpopulation vulnerabilities.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。