arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2410.01423cs.LGcs.AI
收藏

Fair4Free:利用无数据蒸馏生成高保真公平合成样本

Fair4Free: Generating High-fidelity Fair Synthetic Samples using Data Free Distillation

Linköping University(林雪平大学)

Md Fahim Sikder, Daniel de Leng, Fredrik Heintz

AI总结:

针对私有或不可访问数据场景,本文提出 Fair4Free,通过教师模型学习公平表示并无数据蒸馏到小型学生模型来生成公平合成样本,在公平性、效用和合成质量上均超越现有方法。

摘要:

本工作提出了 Fair4Free,这是一种在潜在空间中使用无数据蒸馏来生成公平合成数据的新型生成模型。Fair4Free 可在数据属于私有或无法访问的情形下工作。在我们的方法中,首先训练一个教师模型以创建公平表示,随后将知识蒸馏到一个学生模型(采用更小的架构)。蒸馏学生模型的过程是无数据的,即学生模型在蒸馏期间无法访问训练数据集。蒸馏之后,我们使用经过蒸馏的模型生成公平合成样本。大量实验表明,对于表格和图像数据集,我们的合成样本在公平性、效用和合成质量三项标准上均优于最先进模型,其中公平性性能提升5%,效用提升8%,合成质量提升12%。

英文摘要:

This work presents Fair4Free, a novel generative model to generate synthetic fair data using data-free distillation in the latent space. Fair4Free can work on the situation when the data is private or inaccessible. In our approach, we first train a teacher model to create fair representation and then distil the knowledge to a student model (using a smaller architecture). The process of distilling the student model is data-free, i.e. the student model does not have access to the training dataset while distilling. After the distillation, we use the distilled model to generate fair synthetic samples. Our extensive experiments show that our synthetic samples outperform state-of-the-art models in all three criteria (fairness, utility and synthetic quality) with a performance increase of 5% for fairness, 8% for utility and 12% in synthetic quality for both tabular and image datasets.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑