一种基于注意力的公平对比学习框架
An Attention-based Framework for Fair Contrastive Learning
AI总结：
本文提出一种基于注意力机制的公平对比学习方法，通过建模偏差交互并聚焦去偏样本，在不降低下游准确率的前提下显著提升表示的去偏效果。
摘要：
对比学习已被证明在学习数据的无偏表示方面具有重要作用，尤其是在以高基数和高维敏感信息为特征的复杂环境中。然而，该场景下的现有方法需要对引起偏差的交互进行预定义建模假设，这限制了模型学习去偏表示的能力。在本工作中，我们提出了一种新的公平对比学习方法，该方法采用注意力机制来建模引起偏差的交互，从而能够学习更公平且语义更丰富的嵌入空间。特别地，我们的注意力机制会避开那些混淆模型的引起偏差样本，并聚焦于有助于学习语义有意义表示的去偏样本。我们在公平对比学习中针对现有基线验证了所提方法的优势，并表明我们的方法能够在不牺牲下游准确率的情况下显著提升学习表示中的偏差去除效果。
英文摘要：
Contrastive learning has proven instrumental in learning unbiased representations of data, especially in complex environments characterized by high-cardinality and high-dimensional sensitive information. However, existing approaches within this setting require predefined modelling assumptions of bias-causing interactions that limit the model's ability to learn debiased representations. In this work, we propose a new method for fair contrastive learning that employs an attention mechanism to model bias-causing interactions, enabling the learning of a fairer and semantically richer embedding space. In particular, our attention mechanism avoids bias-causing samples that confound the model and focuses on bias-reducing samples that help learn semantically meaningful representations. We verify the advantages of our method against existing baselines in fair contrastive learning and show that our approach can significantly boost bias removal from learned representations without compromising downstream accuracy.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
