arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2411.02018cs.CLcs.AI
收藏

上下文学习中的捷径学习:综述

Shortcut Learning in In-Context Learning: A Survey

College of Computer Science and Technology, Jilin University(吉林大学计算机科学与技术学院);University of Trento(特伦托大学)

Rui Song, Yingji Li, Lida Shi, Fausto Giunchiglia, Hao Xu

AI总结:

本文从新视角综述上下文学习(ICL)中的捷径学习研究,探讨其类型、成因、基准及缓解策略,总结未解决问题并勾勒未来研究图景,为LLMs的泛化性与鲁棒性提升提供参考。

摘要:

捷径学习指模型在实际任务中采用简单、非鲁棒决策规则的现象,阻碍其泛化性与鲁棒性。近年来,随着大语言模型(LLMs)的快速发展,越来越多研究表明捷径学习对LLMs存在影响。本文从新视角综述上下文学习(ICL)中捷径学习的相关研究,详细探讨ICL任务中的捷径类型、成因、现有基准及缓解策略,并基于观察总结现有研究的未解决问题,尝试勾勒该领域的未来研究图景。

英文摘要:

Shortcut learning refers to the phenomenon where models employ simple, non-robust decision rules in practical tasks, which hinders their generalization and robustness. With the rapid development of large language models (LLMs) in recent years, an increasing number of studies have shown the impact of shortcut learning on LLMs. This paper provides a novel perspective to review relevant research on shortcut learning in In-Context Learning (ICL). It conducts a detailed exploration of the types of shortcuts in ICL tasks, their causes, available benchmarks, and strategies for mitigating shortcuts. Based on corresponding observations, it summarizes the unresolved issues in existing research and attempts to outline the future research landscape of shortcut learning.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑