In-Context Learning (ICL) enables pretrained LLMs to adapt to downstream tasks by conditioning on a small set of input-output demonstrations, without any parameter updates. Although there have been many theoretical efforts to explain how ICL works, most either rely on strong architectural or data assumptions, or fail to capture the impact of key practical factors such as demonstration selection, Chain-of-Thought (CoT) prompting, the number of demonstrations, and prompt templates. We address this gap by establishing a theoretical analysis of ICL under mild assumptions that links these design choices to generalization behavior. We derive an upper bound on the ICL test loss, showing that performance is governed by (i) the quality of selected demonstrations, quantified by Lipschitz constants of the ICL loss along paths connecting test prompts to pretraining samples, (ii) an intrinsic ICL capability of the pretrained model, and (iii) the degree of distribution shift. Within the same framework, we analyze CoT prompting as inducing a task decomposition and show that it is beneficial when demonstrations are well chosen at each substep and the resulting subtasks are easier to learn. Finally, we characterize how ICL performance sensitivity to prompt templates varies with the number of demonstrations. Together, our study shows that pretraining equips the model with the ability to generalize beyond observed tasks, while CoT enables the model to compose simpler subtasks into more complex ones, and demonstrations and instructions enable it to retrieve similar or complex tasks, including those that can be composed into more complex ones, jointly supporting generalization to unseen tasks. All theoretical insights are corroborated by experiments.
Enhancing Granular Sentiment Classification with Chain-of-Thought Prompting in Large Language Models
John F Kennedy High school Fremont, USA(约翰·F·肯尼迪高中弗里蒙特分校);Queensland Academy for Science Mathematics and Technology(昆士兰科学数学与技术学院);Research Spark Hub Inc(Research Spark Hub公司)
Enhance Multi-domain Sentiment Analysis of Review Texts through Prompting Strategies
Hong Kong Baptist University(香港浸会大学);BNU-UIC Institute of AI and Future Networks(北师大-联合国际学院人工智能与未来网络研究院);BNU-HKBU United International College(北师大-港浸大联合国际学院);Beijing Normal University(北京师范大学)
Evaluating LLMs and Prompting Strategies for Automated Hardware Diagnosis from Textual User-Reports
Laboratório de Sistemas e Banco de Dados (LSBD) Departamento de Computação / Universidade Federal do Ceará (UFC) -- Fortaleza -- CE -- Brazil(系统与数据库实验室(LSBD)计算机系/北卡罗来纳州立大学(UFC)——福塔莱扎——CE——巴西)
Advancing Multi-Step Mathematical Reasoning in Large Language Models through Multi-Layered Self-Reflection with Auto-Prompting
Luiz de Queiroz College of Agriculture University of São Paulo(圣保罗大学农业学院);School of Engineering and Sciences Tecnologico de Monterrey(蒙特雷技术学院工程与科学学院);Institute of Mathematics and Computer Sciences University of São Paulo(圣保罗大学数学与计算机科学研究所)
PathCoT: Chain-of-Thought Prompting for Zero-shot Pathology Visual Reasoning
The College of Artificial Intelligence, Nanjing University of Aeronautics(人工智能学院,南京航空航天大学);Astronautics The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education(航空航天脑机智能技术重点实验室,教育部)