金子未必总发光:线性与非线性受保护属性信息的谱移除方法
Gold Doesn't Always Glitter: Spectral Removal of Linear and Nonlinear Guarded Attribute Information
AI总结:
本文提出SAL谱属性移除方法,通过矩阵分解及核推广从神经表示中去除线性与非线性受保护信息,在保护隐私的同时更好保留主任务性能,并降低对敏感标注数据的需求。
摘要:
我们描述了一种简单而有效的方法(Spectral Attribute removaL;SAL),用于从神经表示中移除私有或受保护信息。该方法利用矩阵分解,将输入表示投影到与受保护信息协方差降低的方向上,而不是像分解方法通常所做的那样投影到协方差最大的方向。我们首先从线性信息移除入手,随后利用核函数将算法推广到非线性信息移除情形。实验表明,与先前工作相比,我们的算法在移除受保护信息后能够更好地保留主任务性能。此外,实验还表明,只需相对少量的受保护属性数据即可移除关于这些属性的信息,从而降低敏感数据暴露程度,更适合低资源场景。代码位于 https://github.com/jasonshaoshun/SAL。
英文摘要:
We describe a simple and effective method (Spectral Attribute removaL; SAL) to remove private or guarded information from neural representations. Our method uses matrix decomposition to project the input representations into directions with reduced covariance with the guarded information rather than maximal covariance as factorization methods normally use. We begin with linear information removal and proceed to generalize our algorithm to the case of nonlinear information removal using kernels. Our experiments demonstrate that our algorithm retains better main task performance after removing the guarded information compared to previous work. In addition, our experiments demonstrate that we need a relatively small amount of guarded attribute data to remove information about these attributes, which lowers the exposure to sensitive data and is more suitable for low-resource scenarios. Code is available at https://github.com/jasonshaoshun/SAL.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。