In-Context Learning (ICL) empowers Large Language Models (LLMs) with the ability to learn from a few examples provided in the prompt, enabling downstream generalization without the requirement for gradient updates. Despite encouragingly empirical success, the underlying mechanism of ICL remains unclear. Existing research remains ambiguous with various viewpoints, utilizing intuition-driven and ad-hoc technical solutions to interpret ICL. In this paper, we leverage a data generation perspective to reinterpret recent efforts from a systematic angle, demonstrating the potential broader usage of these popular technical solutions. For a conceptual definition, we rigorously adopt the terms of skill recognition and skill learning. Skill recognition selects one learned data generation function previously seen during pre-training while skill learning can learn new data generation functions from in-context data. Furthermore, we provide insights into the strengths and weaknesses of both abilities, emphasizing their commonalities through the perspective of data generation. This analysis suggests potential directions for future research.
Large Language Model based Multi-Agents: A Survey of Progress and Challenges
University of Notre Dame(圣母大学);King Abdullah University of Science and Technology(阿卜杜拉国王科技大学);Southern University of Science and Technology(南方科技大学);University of Massachusetts Boston(马萨诸塞大学波士顿分校)
A survey on fairness of large language models in e-commerce: progress, application, and challenge
Carnegie Mellon University(卡内基梅隆大学);San Francisco State University(旧金山州立大学);Columbia University(哥伦比亚大学);University of Southern California(南加州大学);Virginia Tech(弗吉尼亚理工大学);Khoury College of Computer Sciences, Northeastern University(东北大学科里计算机科学学院);New York University(纽约大学)
A Survey of Code-switched Arabic NLP: Progress, Challenges, and Future Directions
MBZUAI(Mohamed bin Zayed 人工智能大学);German International University, Egypt(德国国际大学);University of Stuttgart(斯图加特大学);University of Houston(休斯顿大学);New York University Abu Dhabi(纽约大学阿布扎比分校)
A Survey of Arabic Dialogues Understanding for Spontaneous Dialogues and Instant Message
Institute of Statistical Studies and Research (ISSR), Cairo University(开罗大学统计研究与研究所);Faculty of Computers and Information, Cairo University(开罗大学计算机与信息学院)
Progress Reward Modeling for Robotic Learning: A Comprehensive Survey
Northwestern University(西北大学);Carnegie Mellon University(卡内基梅隆大学);University of Wisconsin–Madison(威斯康星大学麦迪逊分校);University of California, Santa Barbara(加利福尼亚大学圣塔芭芭拉分校);University of Illinois Urbana-Champaign(伊利诺伊大学厄巴纳-香槟分校)
Center for Information and Language Processing, LMU Munich(慕尼黑大学信息与语言处理中心);Munich Centre for Machine Learning (MCML)(慕尼黑机器学习中心);Faculty of Mathematics and Physics, Charles University(查理大学数学与物理学院);Technical University of Munich(慕尼黑工业大学)
Progress Note Understanding -- Assessment and Plan Reasoning: Overview of the 2022 N2C2 Track 3 Shared Task
University of Wisconsin Madison(威斯康星大学麦迪逊分校);Loyola University Chicago(芝加哥洛约拉大学);Boston Children’s Hospital(波士顿儿童医院);Harvard University(哈佛大学);George Mason University(乔治梅森大学)