Shortcut learning refers to the phenomenon where models employ simple, non-robust decision rules in practical tasks, which hinders their generalization and robustness. With the rapid development of large language models (LLMs) in recent years, an increasing number of studies have shown the impact of shortcut learning on LLMs. This paper provides a novel perspective to review relevant research on shortcut learning in In-Context Learning (ICL). It conducts a detailed exploration of the types of shortcuts in ICL tasks, their causes, available benchmarks, and strategies for mitigating shortcuts. Based on corresponding observations, it summarizes the unresolved issues in existing research and attempts to outline the future research landscape of shortcut learning.
Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models
University of Science and Technology of China(中国科学技术大学);State Key Lab of Cognitive Intelligence(认知智能国家重点实验室);School of Computer Science and Technology(计算机科学与技术学院)
Investigating the Influence of Prompt-Specific Shortcuts in AI Generated Text Detection
Seoul National University(首尔大学);Hanyang University(汉阳大学);Ewha Womans University(梨花女子大学);NAVER Search US(NAVER美国搜索部门);NAVER AI LAB(NAVER AI实验室);NAVER Cloud(NAVER云)
Sorbonne Université(索邦大学);CNRS(法国国家科学研究中心);LIP6(巴黎第六大学计算机科学实验室);Carney Institute for Brain Science, Brown University(布朗大学卡尼脑科学研究所);Idiap Research Institute(伊迪亚普研究所);Australian Institute for Machine Learning, University of Adelaide(阿德莱德大学澳大利亚机器学习研究所)
Sparse Shortcuts: Facilitating Efficient Fusion in Multimodal Large Language Models
Guangdong-Hong Kong-Macao Joint Laboratory for Emotion Intelligence and Pervasive Computing, Artificial Intelligence Research Institute, Shenzhen MSU-BIT University, Shenzhen(粤港澳大湾区情感智能与 pervasive 计算联合实验室,人工智能研究院,深圳MSU-BIT大学)
Shortcut Learning in Legal Judgment Prediction: Empirical Evidence from the UK Employment Tribunal
Faculty of Law, University of Cambridge(剑桥大学法学院);The Psychometrics Centre, Cambridge Judge Business School, University of Cambridge(剑桥大学心理学测量中心,剑桥Judge商学院);Faculty of English, University of Cambridge(剑桥大学英语学院);Department of Statistics, Uppsala University(乌普萨拉大学统计系);Department of Computer Science and Technology, Tsinghua University(清华大学计算机科学与技术系);Department of Engineering, University of Cambridge(剑桥大学工程系)
Beyond the Parameters: A Technical Survey of Contextual Enrichment in Large Language Models: From In-Context Prompting to Causal Retrieval-Augmented Generation
Causal Path Alignment: Anchoring the Optimization Trajectory for Controllable In-Parameter Knowledge Editing
Institute of Information Engineering, Chinese Academy of Sciences(中国科学院信息工程研究所);School of Cyber Security, University of Chinese Academy of Sciences(中国科学院大学网络安全学院)
Beyond the Sentence: A Survey on Context-Aware Machine Translation with Large Language Models
Department of Computer Science and Engineering, Indian Institute of Technology Patna, India(印度理工学院帕纳瓦分校计算机科学与工程系);Wipro AI, Lab45, London, UK(Wipro AI,Lab45,伦敦,英国);School of AI and Data Science, Indian Institute of Technology Jodhpur, India(印度理工学院朱罗普分校人工智能与数据科学学院)
Peking University(北京大学);The Hong Kong Polytechnic University(香港理工大学);ByteDance(字节跳动);Shanghai AI Lab(上海人工智能实验室);Alibaba Group(阿里巴巴集团);Carnegie Mellon University(卡内基梅隆大学)
A Survey of Context Engineering for Large Language Models
Institute of Computing Technology, Chinese Academy of Sciences(中国科学院计算技术研究所);University of California, Merced(加州大学默塞德分校);The University of Queensland(昆士兰大学);Peking University(北京大学);Tsinghua University(清华大学);University of Chinese Academy of Sciences(中国科学院大学)
CARO: Chain-of-Analogy Reasoning Optimization for Robust Content Moderation
National Engineering Laboratory for Big Data System Computing Technology(大数据系统计算技术国家工程实验室);Shenzhen University(深圳大学);Tsinghua University(清华大学);National University of Singapore(新加坡国立大学)
The What, Why, and How of Context Length Extension Techniques in Large Language Models -- A Detailed Survey
Technology Innovation Institute(技术创新研究所);Islamic University of Technology(伊斯兰理工大学);Stanford University(斯坦福大学);Amazon GenAI(亚马逊生成式人工智能部门);AI Institute, University of South Carolina(南卡罗来纳大学人工智能研究所)
Semformer: Transformer Language Models with Semantic Planning
Zhejiang University(浙江大学);School of Engineering, Westlake University(西湖大学工学院);ByteDance(字节跳动);Institute of Advanced Technology, Westlake Institute for Advanced Study(西湖高等研究院先进技术研究所)