导航捷径迷宫:语言模型在文本分类中捷径学习的全面分析
Navigating the Shortcut Maze: A Comprehensive Analysis of Shortcut Learning in Text Classification by Language Models
AI总结:
本研究分析了语言模型在文本分类中对复杂捷径的依赖问题,提出了一个将捷径分为出现、风格和概念三类的全面基准,并通过多类模型的广泛实验系统评估了模型对复杂捷径的韧性与易感性。
摘要:
语言模型(LMs)尽管取得了进展,但往往依赖虚假相关性,削弱了其准确性和泛化能力。本研究探讨了更微妙、更复杂的捷径对模型可靠性的被忽视的影响,这些捷径超越了过度简化的捷径。我们引入了一个全面的基准,将捷径分为出现、风格和概念三类,旨在探索这些捷径影响LMs性能的细微方式。通过对传统LMs、大型语言模型和最先进的鲁棒模型进行广泛实验,我们的研究系统地调查了模型对复杂捷径的韧性和易感性。我们的基准和代码可在以下网址找到:https://github.com/yuqing-zhou/shortcut-learning-in-text-classification。
英文摘要:
Language models (LMs), despite their advances, often depend on spurious correlations, undermining their accuracy and generalizability. This study addresses the overlooked impact of subtler, more complex shortcuts that compromise model reliability beyond oversimplified shortcuts. We introduce a comprehensive benchmark that categorizes shortcuts into occurrence, style, and concept, aiming to explore the nuanced ways in which these shortcuts influence the performance of LMs. Through extensive experiments across traditional LMs, large language models, and state-of-the-art robust models, our research systematically investigates models' resilience and susceptibilities to sophisticated shortcuts. Our benchmark and code can be found at: https://github.com/yuqing-zhou/shortcut-learning-in-text-classification.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。