arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2202.13576cs.LGcs.ITmath.ITstat.ML
收藏

多组属性下的KL散度估计

KL Divergence Estimation with Multi-group Attribution

VMware Research(威睿研究);Stanford University(斯坦福大学)

Parikshit Gopalan, Nina Narodytska, Omer Reingold, Vatsal Sharan, Udi Wieder

摘要:

在机器学习和信息论中,对给定样本的两个分布之间的Kullback-Leibler(KL)偏差进行了深入研究。出于对多组公平性的考虑,我们寻求KL分歧估计,准确地反映了子群体的整体分歧的贡献。我们模拟的子群体来自一个丰富的(可能是无限的)家庭$\mathcal{C}$重叠的子集的域。我们提出了$\mathcal{C}$的多组归因的概念,这要求在$\mathcal{C}$中的每个子群体的条件下,估计的分歧满足一些自然的准确性和公平性的要求,如确保子群体的模型预测显着分歧显着分歧显着的两个分布。我们的主要技术贡献是表明,多组属性可以从最近引入的重要性权重多重校准的概念中推导出来[HKRR18,GRSW 21]。我们提供了实验证据来支持我们的理论结果,并表明,多组归因提供了更好的KL分歧估计时,比其他流行的算法的子种群的条件。

英文摘要:

Estimating the Kullback-Leibler (KL) divergence between two distributions given samples from them is well-studied in machine learning and information theory. Motivated by considerations of multi-group fairness, we seek KL divergence estimates that accurately reflect the contributions of sub-populations to the overall divergence. We model the sub-populations coming from a rich (possibly infinite) family $\mathcal{C}$ of overlapping subsets of the domain. We propose the notion of multi-group attribution for $\mathcal{C}$, which requires that the estimated divergence conditioned on every sub-population in $\mathcal{C}$ satisfies some natural accuracy and fairness desiderata, such as ensuring that sub-populations where the model predicts significant divergence do diverge significantly in the two distributions. Our main technical contribution is to show that multi-group attribution can be derived from the recently introduced notion of multi-calibration for importance weights [HKRR18, GRSW21]. We provide experimental evidence to support our theoretical results, and show that multi-group attribution provides better KL divergence estimates when conditioned on sub-populations than other popular algorithms.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑