arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2605.16645math.STcs.ITcs.LGmath.ITstat.MLstat.TH
收藏

分布统计遗忘:一种假设检验方法

Statistical Unlearning of Distributions: A Hypothesis Testing Approach

Princeton University(普林斯顿大学)

Aaradhya Pandey, Sanjeev Kulkarni

AI总结:

本文提出一种分布统计遗忘框架,通过假设检验选择样本以减少不需要的分布影响,同时保持所需分布的性能,并分析了允许的编辑数据分布区域和帕累托前沿。

摘要:

机器学习系统越来越多地面临要求遗忘不仅单个数据点,还包括整个信息领域的需求,例如有毒语言、受版权保护的语料库或人口统计数据偏见。这提出了统计-计算权衡的根本困境:移除所有不需要领域的样本可能是计算上不可行的,而随机移除一部分可能无法提供分布层面的统计保证。我们提出了一种分布遗忘的统计框架,其中领域被建模为概率分布,目标是移除精心选择的样本子集,以减少不需要分布的影响,同时保持所需分布的性能。我们通过假设检验编辑数据与所需和不需要的领域,从而得到可解释且稳健的样本移除标准。在该统计框架中,我们表征了允许的编辑数据分布区域以及广泛分布族的移除-保留帕累托前沿。这包括参数族如任意维度的位移高斯分布、一维位置族带有对数凹噪声以及一维泊松族。它还包含非参数族,如高斯白噪声模型,这是非参数回归的通用模型。我们证明了组合规则,描述了分布遗忘在多模式不需要领域中的行为,并引入了当组合大量此类族时移除-保留基线的中心极限行为。最后,我们通过提供某些选择算法的帕累托前沿来提供有限样本保证,并观察到信息-计算差距。

英文摘要:

Machine learning systems increasingly face requirements to forget not only individual data points, but entire domains of information, such as toxic language, copyrighted corpora, or demographic biases. This raises a fundamental dilemma of statistical-computational tradeoffs: removing all samples from an unwanted domain may be computationally prohibitive, while randomly removing a subset may not provide distribution-level statistical guarantees. We propose a statistical framework for distributional unlearning, in which domains are modeled as probability distributions, and the goal is to remove a carefully chosen subset of samples that reduces the effect of an unwanted distribution while preserving performance on a desired one. We formalize this using a hypothesis test of the edited data with the desired and unwanted domains, leading to an interpretable and robust criterion for selecting samples to remove. Within this statistical framework, we characterize the fundamental region of the allowable edited data distributions and the removal-preservation Pareto frontier for a broad class of distribution families. This includes parametric families such as shifted Gaussians of arbitrary dimension, a one-dimensional location family with log-concave noise, and the one-dimensional Poisson family. It also includes nonparametric families such as the Gaussian white noise model, a canonical model for nonparametric regression. We prove composition rules that describe how distributional unlearning behaves across multimodal unwanted domains, and introduce a central-limit behavior for the removal-preservation baselines when composing a large number of such families. Finally, we provide finite sample guarantees by providing Pareto frontiers for some selection algorithms, and observe an information-computation gap.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑