Decoder-only language models have the ability to dynamically switch between various computational tasks based on input prompts. Despite many successful applications of prompting, there is very limited understanding of the internal mechanism behind such flexibility. In this work, we investigate how different prompting methods affect the geometry of representations in these models. Employing a framework grounded in statistical physics, we reveal that various prompting techniques, while achieving similar performance, operate through distinct representational mechanisms for task adaptation. Our analysis highlights the critical role of input distribution samples and label semantics in few-shot in-context learning. We also demonstrate evidence of synergistic and interfering interactions between different tasks on the representational level. Our work contributes to the theoretical understanding of large language models and lays the groundwork for developing more effective, representation-aware prompting strategies.
AdaRewriter: Unleashing the Power of Prompting-based Conversational Query Reformulation via Test-Time Adaptation
School of Computer Science and Engineering, Key Laboratory of Computer Network and Information Integration, Ministry of Education, Southeast University(计算机科学与工程学院、计算机网络与信息集成重点实验室、教育部、东南大学)
Transfer-Prompting: Enhancing Cross-Task Adaptation in Large Language Models via Dual-Stage Prompts Optimization
School of Artificial intelligence, Jilin University(吉林大学人工智能学院);Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University(吉林大学知识驱动人机智能工程研究中心);International Center of Future Science, Jilin University(吉林大学未来科学国际中心)
Beyond Surface Reasoning: Unveiling the True Long Chain-of-Thought Capacity of Diffusion Large Language Models
LARG, Research Center for Social Computing and Interactive Robotics, Harbin Institute of Technology(LARG,社会计算与交互机器人研究中心,哈尔滨工业大学);School of Computer Science and Engineering, Central South University(计算机科学与工程学院,中南大学);The University of Hong Kong(香港大学);Shanghai Jiao Tong University(上海交通大学);ByteDance Seed (China)(字节跳动种子(中国))
Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning
Department of Computing, The Hong Kong Polytechnic University(香港理工大学计算机系);Ningbo Key Laboratory of Spatial Intelligence and Digital Derivative(宁波空间智能与数字衍生关键实验室);Institute of Digital Twin, EIT(数字孪生研究院);Saarland University(萨尔兰大学);Meituan Inc.(美团公司)
Unveiling and Mitigating Bias in Mental Health Analysis with Large Language Models
Stanford University(斯坦福大学);Meta Platforms, Inc.(元平台公司);ETH Zürich(苏黎世联邦理工学院);University Medical Center Utrecht(乌得勒支大学医学中心);Leiden University Medical Center(莱顿大学医学中心)
Unveiling Factual Recall Behaviors of Large Language Models through Knowledge Neurons
Institute of Automation, Chinese Academy of Sciences(中国科学院自动化研究所);University of Chinese Academy of Sciences(中国科学院大学);Beijing Wenge Technology Co., Ltd(北京文格科技有限公司)
NCL-UoR at SemEval-2026 Task 5: Embedding-Based Methods, Fine-Tuning, and LLMs for Word Sense Plausibility Rating
Independent Researcher(独立研究者);Department of Computer Science, University of Reading(阅读大学计算机科学系);School of Computing, Newcastle University(新castle大学计算学院)
DECASTE: Unveiling Caste Stereotypes in Large Language Models through Multi-Dimensional Bias Analysis
IBM Research, San Jose, CA, USA(IBM研究院,美国加利福尼亚州圣何塞);Dartmouth College, Hanover, NH, USA(达特茅斯学院,美国新罕布什尔州汉诺威);IBM Research, Hursley, Winchester, UK(IBM研究院,英国赫里福德郡温切斯特);IBM Thomas J. Watson Research Center, Yorktown Heights, NY, USA(IBM托马斯·J·沃森研究中心,美国纽约州亚当斯);IBM Research, Sao Paulo, Brazil(IBM研究院,巴西圣保罗)
PromptEmbedder: Efficient and Transferable Text Embedding via Dual-LLM Soft Prompting
Department of Computer Science and Information Engineering, National Taiwan University(国立台湾大学计算机科学与资讯工程系);National Taiwan University AI Center of Research Excellence(国立台湾大学人工智能研究中心)
On Conditional and Compositional Language Model Differentiable Prompting
Mila - Québec AI Institute(米拉-魁北克人工智能研究所);Polytechnique Montréal(蒙特利尔理工学院);Amazon Alexa(亚马逊Alexa);University of North Carolina at Chapel Hill(北卡罗来纳大学教堂山分校)
BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting
Brown University(布朗大学);EleutherAI;Yale University(耶鲁大学);Hugging Face;MBZUAI(穆罕默德·本·扎耶德人工智能大学);University College London(伦敦大学学院);PSAU(沙特王子萨塔姆·本·阿卜杜勒阿齐兹大学);Nanyang Technological University(南洋理工大学);University of Washington(华盛顿大学);University of Oregon(俄勒冈大学);The Hong Kong University of Science and Technology(香港科技大学);Booz Allen Hamilton(博思艾伦咨询公司);NAVER LABS Europe(NAVER欧洲实验室)
From Weights to Activations: Is Steering the Next Frontier of Adaptation?
Saarland University(萨尔兰大学);German Research Center for Artificial Intelligence (DFKI)(德国人工智能研究中心);Centre for European Research in Trusted AI (CERTAIN)(可信AI欧洲研究中心);Munich Center for Machine Learning(慕尼黑机器学习中心);Fraunhofer Heinrich Hertz Institute(弗劳恩霍夫海因里希·赫兹研究所);Technological University Dublin(都柏林技术大学);Technische Universität Berlin(柏林技术大学);Berlin Institute for the Foundations of Learning and Data (BIFOLD)(柏林学习与数据基础研究所)
Clinical information extraction for Low-resource languages with Few-shot learning using Pre-trained language models and Prompting
Klaus Tschira Institute for Integrative Computational Cardiology(克劳斯·齐拉整合计算心脏病学研究所);Heidelberg University(海德堡大学);University Hospital Heidelberg(海德堡大学医院);German Center for Cardiovascular Research - Partner site Heidelberg/Mannheim(德国心血管研究中心海德堡/曼海姆合作站点);Informatics for Life(生命信息学机构)