Chameleon:用于公平感知多模态数据增强以提升少数群体覆盖率的基础模型
Chameleon: Foundation Models for Fairness-aware Multi-modal Data Augmentation to Enhance Coverage of Minorities
AI总结:
针对训练数据中少数群体代表性不足的问题,提出Chameleon系统,利用基础模型通过拒绝采样和指导策略生成高质量合成元组增强数据,有效提升少数群体覆盖率并降低下游任务模型不公平性。
摘要:
训练数据中少数群体代表性不足的潜在危害,尤其是在多模态场景下,是一个公认的问题。尽管在检测此类代表性不足方面已有大量努力,但解决这一问题仍具挑战。随着生成式AI的最新进展,大型语言模型和基础模型已成为跨多个领域的多功能工具。在本文中,我们提出Chameleon系统,该系统高效利用这些工具,通过添加最少的合成生成元组来增强数据集,以提升代表性不足群体的覆盖率。我们的系统采用拒绝采样方法,确保生成的元组具有高质量并遵循潜在分布。为了最小化生成元组的拒绝概率,我们提出多种策略为基础模型提供指导。我们的实验结果不仅证实了所提算法的效率,还表明在使用Chameleon进行数据修复后,下游任务中模型的不公平性显著下降。
英文摘要:
The potential harms of the under-representation of minorities in training data, particularly in multi-modal settings, is a well-recognized concern. While there has been extensive effort in detecting such under-representation, resolution has remained a challenge. With recent advancements in generative AI, large language models and foundation models have emerged as versatile tools across various domains. In this paper, we propose Chameleon, a system that efficiently utilizes these tools to augment a data set with a minimal addition of synthetically generated tuples, in order to enhance the coverage of the under-represented groups. Our system follows a rejection sampling approach to ensure the generated tuples have a high quality and follow the underlying distribution. In order to minimize the rejection chance of the generated tuples, we propose multiple strategies for providing a guide for the foundation model. Our experiment results, in addition to confirming the efficiency of our proposed algorithms, illustrate the effectiveness of our approach, as the unfairness of the model in a downstream task significantly dropped after data repair using Chameleon.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。