In-context learning (ICL) excels at new tasks from minimal examples, yet we still lack a mechanistic explanation of how few-shot prompts shape a model's function vector (FV)--a causal activation direction that drives task behavior on the ICL query. Across tasks and models, an $n$-shot FV is well-approximated by a linear combination of example-level sub-FVs, suggesting additive and composable contributions from individual demonstrations. Beyond additivity, we show that models contextualize individual examples' representations based on prior examples to adaptively reweight which demonstrations dominate the FV: attention shifts toward examples that are more informative and less ambiguous under the context. Finally, a causal decomposition separates Query-Key routing from Value updates, finding that contextualization's most consistent contributions to FV quality arise from Query-Key alignment--particularly in ambiguous settings--while Value-mediated effects are more heterogeneous. Together, these results unify additive superposition with context-dependent attention reweighting into a mechanistic, testable account of how few-shot prompts implement tasks.
Revisiting Few-Shot Learning from a Causal Perspective
School of Computer Science and Engineering, Sun Yat-Sen University(中山大学计算机科学与工程学院);School of Artificial Intelligence, Sun Yat-Sen University(中山大学人工智能学院)
Prediction and Causality of functional MRI and synthetic signal using a Zero-Shot Time-Series Foundation Model
AGH University of Krakow, Poland(克拉科夫AGH大学);Institut de Neurosciences de la Timone UMR 7289, Aix Marseille Université, CNRS, 13005, Marseille, France(里莫内神经科学研究所)
IDIAPers @ Causal News Corpus 2022: Efficient Causal Relation Identification Through a Prompt-based Few-shot Approach
Idiap Research Institute(Idiap研究所);Universidad Nacional de San Luis (UNSL)(圣路易斯国立大学);Ecole Polytechnique Fédérale de Lausanne(洛桑联邦理工学院);Universidad Autónoma Metropolitana Unidad Cuajimalpa(墨西哥都会自治大学夸伊马尔帕分校);Brno University of Technology(布尔诺理工大学)
Inducing Causal World Models in LLMs for Zero-Shot Physical Reasoning
Department of Electrical Engineering(电气工程系);Indian Institute of Technology Bombay(印度理工学院孟买分校);Department of Computer Science and Automation(计算机科学与自动化系);Indian Institute of Science(印度科学研究所);Department of Computer Science(计算机科学系);San Francisco State University(旧金山州立大学);University of Lagos(拉各斯大学);Faculty of Mathematics, Informatics, and Mechanics(数学、信息学与力学学院);University of Warsaw(华沙大学)
Mutually Causal Semantic Distillation Network for Zero-Shot Learning
Huazhong University of Science and Technology(华中科技大学);National Anti-Counterfeit Engineering Research Center(国家防伪工程技术研究中心);Nanjing University of Science and Technology(南京理工大学)
Semantic4Safety: Causal Insights from Zero-shot Street View Imagery Segmentation for Urban Road Safety
School of Geospatial Engineering and Science, Sun Yat-sen University(地理空间工程与科学学院,中山大学);School of Geographical and Earth Sciences, University of Glasgow(地理与地球科学学院,格拉斯哥大学);School of Economics and Management, Shanxi University(经济学与管理学院,山西大学)
CMVAE: Causal Meta VAE for Unsupervised Meta-Learning
College of Information Science and Electronic Engineering, Zhejiang University(浙江大学信息与电子工程学院);ZJU-League Research & Development Center(浙大-联队研发中心);State Key Lab of CAD&CG, Zhejiang University(浙江大学CAD&CG国家重点实验室);Zhejiang Provincial Key Laboratory of Information Processing, Communication and Networking(浙江省信息处理、通信与网络重点实验室)
MapPFN: Learning Causal Perturbation Maps in Context
Machine Learning Group, Technische Universität Berlin(技术大学柏林机器学习组);Berlin Institute for the Foundations of Learning and Data (BIFOLD)(柏林学习与数据基础研究院(BIFOLD));Aignostics, Berlin, Germany(柏林Aignostics公司);Institute of Pathology, Charité - Universitätsmedizin Berlin(柏林Charité大学医学中心病理研究所)
Causal Reasoning Favors Encoders: On The Limits of Decoder-Only Models
SIRE, IIT Delhi(IIT德里SIRE);Robert Bosch GmbH, India(印度罗伯特博世集团);IIIT Hyderabad(海得拉巴IIIT);IISER Kolkata(科钦IISER);Microsoft and the University of York(微软和约克大学)
Few-Shot Degradation Is Not What It Seems: Behavioral Evidence, Representation Analysis, and a Random-Text Control Across 12 Models, 2 Tasks, and 2 Architectures
LEX AI Platform, legal.org.ua(LEX AI平台,legal.org.ua)