Prompt engineering is crucial for deploying LLMs but is poorly understood mathematically. We formalize LLM systems as a class of discrete stochastic dynamical systems to explore prompt engineering through the lens of control theory. We offer a mathematical analysis of the limitations on the controllability of self-attention as a function of the singular values of the parameter matrices. We present complementary empirical results on the controllability of a panel of LLMs, including Falcon-7b, Llama-7b, and Falcon-40b. Given initial state $\mathbf x_0$ from Wikitext and prompts of length $k \leq 10$ tokens, we find that the "correct" next token is reachable at least 97% of the time, and that the top 75 most likely next tokens are reachable at least 85% of the time. Intriguingly, short prompt sequences can dramatically alter the likelihood of specific outputs, even making the least likely tokens become the most likely ones. This control-theoretic analysis of LLMs demonstrates the significant and poorly understood role of input sequences in steering output probabilities, offering a foundational perspective for enhancing language model system capabilities.
Jailbreaking LLMs' Safeguard with Universal Magic Words for Text Embedding Models
Dept. of Comp. Sci. and Tech., Inst. for AI, Tsinghua-Bosch Joint ML Center, THBI Lab, BNRist Center, Tsinghua University, Beijing, 100084, China（计算机科学与技术系、人工智能研究所、清华-博世联合机器学习中心、THBI实验室、BNRist中心、清华大学、北京）；Department of Mathematical Sciences, Tsinghua University（数学科学系、清华大学）；BIMSA, Beijing, China（BIMSA、北京、中国）
Large language models (LLMs) demonstrate remarkable machine translation (MT) abilities via prompting, even though they were not explicitly trained for this task. However, even given the incredible quantities of data they are trained on, LLMs can struggle to translate inputs with rare words, which are common in low resource or domain transfer scenarios. We show that LLM prompting can provide an effective solution for rare words as well, by using prior knowledge from bilingual dictionaries to provide control hints in the prompts. We propose a novel method, DiPMT, that provides a set of possible …
Large Language Models are affected by the phenomena of memorizing and forgetting their training data. But how do these vary by model size? We work towards this question by investigating how the model size affects the model's ability to discriminate a word's meaning in a given context. We introduce a dataset called DeltaWords, which evaluates a model's ability to follow instructions to select a sentence which replaces the target word with its antonym. We show a weak inverse scaling trend, where task accuracy degrades as model size increase, under extremely few-shot prompting regimes. We show th…
From Words to Widgets for Controllable LLM Generation
Cornell University（康奈尔大学）；University of Illinois Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）；The University of Texas at Austin（德克萨斯大学奥斯汀分校）；Loyola University Maryland（路易斯安那大学马里兰分校）
NCL-UoR at SemEval-2026 Task 5: Embedding-Based Methods, Fine-Tuning, and LLMs for Word Sense Plausibility Rating
Independent Researcher（独立研究者）；Department of Computer Science, University of Reading（阅读大学计算机科学系）；School of Computing, Newcastle University（新castle大学计算学院）
Leveraging the Potential of Prompt Engineering for Hate Speech Detection in Low-Resource Languages
Bangladesh Institute of Governance and Management (BIGM)（孟加拉国治理与管理研究所）；Department of Cyber Security Engineering, George Mason University（网络安全工程系，乔治·梅森大学）
We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. Th…
Beyond Magic Words: Sharpness-Aware Prompt Evolving for Robust Large Language Models with TARE
University of California, Los Angeles（加州大学洛杉矶分校）；Georgia Institute of Technology（佐治亚理工学院）；University of Chicago（芝加哥大学）；University of Oxford（牛津大学）；Ludwig-Maximilians-Universität München（慕尼黑大学）；Yale University（耶鲁大学）
Meta-Task Prompting Elicits Embeddings from Large Language Models
University of Amsterdam（阿姆斯特丹大学）；University of Maryland（马里兰大学）；University of Technology Sydney（悉尼科技大学）；Microsoft Corporation（微软公司）；Tencent IEG（腾讯互动娱乐事业群）
Probing Pre-trained Language Models (PLMs) using prompts has indirectly implied that language models (LMs) can be treated as knowledge bases. To this end, this phenomena has been effective especially when these LMs are fine-tuned towards not just data of a specific domain, but also to the style or linguistic pattern of the prompts themselves. We observe that, satisfying a particular linguistic pattern in prompts is an unsustainable constraint that unnecessarily lengthens the probing task, especially because, they are often manually designed and the range of possible prompt template patterns ca…
On Conditional and Compositional Language Model Differentiable Prompting
Mila - Québec AI Institute（米拉-魁北克人工智能研究所）；Polytechnique Montréal（蒙特利尔理工学院）；Amazon Alexa（亚马逊Alexa）；University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）
AlignedCoT: Prompting Large Language Models via Native-Speaking Demonstrations
The Hong Kong University of Science and Technology (Guangzhou)（香港科技大学（广州））；City University of Hong Kong（香港城市大学）；Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区）；The Hong Kong University of Science and Technology（香港科技大学）；University of California, Merced（加州大学默塞德分校）；Chongqing University（重庆大学）；MBZUAI（穆罕默德·本·扎耶德人工智能大学）
