CURE：面向鲁棒嵌入的受控遗忘——缓解预训练语言模型中的概念捷径
CURE: Controlled Unlearning for Robust Embeddings -- Mitigating Conceptual Shortcuts in Pre-Trained Language Models
AI总结：
CURE通过解耦并抑制概念捷径，在保留内容信息的同时，利用对比学习可控去偏，在IMDB和Yelp上分别提升F1分数10和2个百分点，实现鲁棒且公平的语言理解。
摘要：
预训练语言模型在各种应用中取得了显著成功，但仍易受虚假的、概念驱动的相关性影响，这些相关性损害了鲁棒性和公平性。在这项工作中，我们引入了CURE，一个新颖且轻量级的框架，它系统地解耦并抑制概念捷径，同时保留关键的内容信息。我们的方法首先通过一个由反转网络强化的专用内容提取器提取与概念无关的表示，确保任务相关信息损失最小。随后，一个可控的去偏模块采用对比学习来精细调整残余概念线索的影响，使模型能够根据目标任务适当地减少有害偏见或利用有益相关性。在IMDB和Yelp数据集上使用三种预训练架构进行评估，CURE在IMDB上实现了F1分数绝对提升+10个百分点，在Yelp上提升+2个百分点，同时引入了最小的计算开销。我们的方法为对抗概念偏见建立了一个灵活、无监督的蓝图，为更可靠和公平的语言理解系统铺平了道路。
英文摘要：
Pre-trained language models have achieved remarkable success across diverse applications but remain susceptible to spurious, concept-driven correlations that impair robustness and fairness. In this work, we introduce CURE, a novel and lightweight framework that systematically disentangles and suppresses conceptual shortcuts while preserving essential content information. Our method first extracts concept-irrelevant representations via a dedicated content extractor reinforced by a reversal network, ensuring minimal loss of task-relevant information. A subsequent controllable debiasing module employs contrastive learning to finely adjust the influence of residual conceptual cues, enabling the model to either diminish harmful biases or harness beneficial correlations as appropriate for the target task. Evaluated on the IMDB and Yelp datasets using three pre-trained architectures, CURE achieves an absolute improvement of +10 points in F1 score on IMDB and +2 points on Yelp, while introducing minimal computational overhead. Our approach establishes a flexible, unsupervised blueprint for combating conceptual biases, paving the way for more reliable and fair language understanding systems.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
