University of Illinois at Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校);University of California, San Diego(加州大学圣地亚哥分校);National University of Singapore(新加坡国立大学);University of California, Los Angeles(加州大学洛杉矶分校);Sri Aurobindo International Centre of Education(斯里阿罗bindo国际教育中心)
Dataset Distillation (DD) aims to generate a compact synthetic dataset that enables models to achieve performance comparable to training on the full large dataset, significantly reducing computational costs. Drawing from optimal transport theory, we introduce WMDD (Wasserstein Metric-based Dataset Distillation), a straightforward yet powerful method that employs the Wasserstein metric to enhance distribution matching.
We compute the Wasserstein barycenter of features from a pretrained classifier to capture essential characteristics of the original data distribution. By optimizing synthetic data to align with this barycenter in feature space and leveraging per-class BatchNorm statistics to preserve intra-class variations, WMDD maintains the efficiency of distribution matching approaches while achieving state-of-the-art results across various high-resolution datasets. Our extensive experiments demonstrate WMDD's effectiveness and adaptability, highlighting its potential for advancing machine learning applications at scale.
Department of Statistics, The Pennsylvania State University(宾夕法尼亚州立大学统计学系);Department of Biostatistics and Bioinformatics, Duke University(杜克大学生物统计学与生物信息学系)
描述模型漂移是无监督学习中的一个开放性问题。很难评估无监督模型在什么时候偏离了来自同一人群的不同样本的预期。这对于没有概率解释的模型尤其如此。拓扑数据分析(Topological Data Analysis),特别是映射器算法(Mapper algorithm),已经在许多领域中得到应用,但是描述映射器图的模型漂移是一个研究不足的领域,因为即使是现有的测量相关结构(如图或单纯复形)之间距离的技术也无法解释映射器图表示拓扑、度量和密度信息的组合。在本文中,我们开发了一个最佳的运输为基础的度量,我们称之为网络增强Wasserstein距离评估映射图之间的距离,并证明了模型漂移分析的度量值,通过使用该度量转换模型漂移问题到一个异常检测问题的动态图。
Model-agnostic bias mitigation methods with regressor distribution control for Wasserstein-based fairness metrics
Discover Financial Services(发现金融服务公司)
这篇文章是我们早期工作Miroshnikov et al.(2021)关于公平性可解释性的姊妹篇,其中介绍了偏见解释。在目前的工作中,我们提出了一个偏见缓解方法的基础上,建设后处理模型,更公平的回归分布的Wasserstein为基础的公平性指标。通过识别对偏差贡献最大的预测因子列表,我们通过减轻源自这些预测因子的偏差来降低问题的维度。后处理方法包括通过平衡正偏差和负偏差解释来重塑预测分布,并允许回归偏差降低。我们设计了一个算法,使用贝叶斯优化构造的偏差性能的有效前沿的家庭后处理模型,从其中选择一个最佳的模型。我们的新方法在低维空间中进行优化,避免了昂贵的模型再训练。
On the application of the Wasserstein metric to 2D curves classification
Department of Optimization and Modelling of Dynamical Systems(优化与动态系统建模系);Systems Research Institute, PAS(系统研究所以及波兰科学院);Faculty of Mathematics and Information Science(数学与信息科学学院);Warsaw University of Technology(华沙理工大学)