Large Language Models (LLMs) have demonstrated remarkable proficiency across diverse tasks, exhibiting emergent properties such as semantic prompt comprehension, In-Context Learning (ICL), and Chain-of-Thought (CoT) reasoning. Despite their empirical success, the theoretical mechanisms driving these phenomena remain poorly understood. This study dives into the foundations of these observations by addressing three critical questions: (1) How do LLMs accurately decode prompt semantics despite being trained solely on a next-token prediction objective? (2) Through what mechanism does ICL facilitate performance gains without explicit parameter updates? and (3) Why do intermediate reasoning steps in CoT prompting effectively unlock capabilities for complex, multi-step problems?
Our results demonstrate that, through the autoregressive process, LLMs are capable of exactly inferring the transition probabilities between tokens across distinct tasks using provided prompts. We show that ICL enhances performance by reducing prompt ambiguity and facilitating posterior concentration on the intended task. Furthermore, we find that CoT prompting activates the model's capacity for task decomposition, breaking complex problems into a sequence of simpler sub-tasks that the model has mastered during the pretraining phase. By comparing their individual error bounds, we provide novel theoretical insights into the statistical superiority of advanced prompt engineering techniques.
Enhancing Granular Sentiment Classification with Chain-of-Thought Prompting in Large Language Models
John F Kennedy High school Fremont, USA（约翰·F·肯尼迪高中弗里蒙特分校）；Queensland Academy for Science Mathematics and Technology（昆士兰科学数学与技术学院）；Research Spark Hub Inc（Research Spark Hub公司）
Thought-Path Contrastive Learning via Premise-Oriented Data Augmentation for Logical Reading Comprehension
School of Computer Science and Technology, Beijing Institute of Technology（北京理工大学计算机科学与技术学院）；Beijing Engineering Research Center of High Volume Language Information Processing and Cloud Computing Applications, Beijing Institute of Technology（北京理工大学海量语言信息处理与云计算应用北京工程研究中心）
"Sorry, Come Again?" Prompting -- Enhancing Comprehension and Diminishing Hallucination with [PAUSE]-injected Optimal Paraphrasing
AI Institute, University of South Carolina（南卡罗来纳大学AI研究院）；Indian Institute of Technology, Kharagpur（印度理工学院克勒格布尔分校）；Islamic University of Technology（伊斯兰科技大学）；Stanford University（斯坦福大学）；Amazon AI（亚马逊AI研究院）
Knowlesdge Lab, University of Chicago, Chicago, Illinois, USA（知识实验室，芝加哥大学，芝加哥，伊利诺伊州，美国）；Knowledge Lab, Data Science Institute, Department of Sociology, University of Chicago, Chicago, Illinois, USA（知识实验室，数据科学学院，社会学系，芝加哥大学，芝加哥，伊利诺伊州，美国）
Soft-Prompting with Graph-of-Thought for Multi-modal Representation Learning
School of Computer Science, Wuhan University（武汉大学计算机学院）；National Engineering Research Center for Multimedia Software（国家多媒体软件工程技术研究中心）；School of Electronic Information Engineering, Henan Polytechnic Institute（河南工业职业技术学院电子信息工程学院）；JD Explore Academy（京东探索研究院）
Beyond Reproduction: A Paired-Task Framework for Assessing LLM Comprehension and Creativity in Literary Translation
Natural Language Learning Group (NLLG) School of Business Informatics and Mathematics University of Mannheim（曼海姆大学自然语言学习组（NLLG）商学院信息与数学学院）；University of Technology Nuremberg (UTN), Department Engineering（纽伦堡技术大学（UTN）工程系）；University of Gent, Department of Translation, Interpreting and Communication（根特大学翻译、口译与传播系）；University of Aberdeen, Department of Computing Science（阿伯丁大学计算科学系）；Natural Language Learning and Generation (NLLG) Lab（自然语言学习与生成（NLLG）实验室）
Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；School of Artificial Intelligence, University of Chinese Academy of Sciences（中国科学院大学人工智能学院）；Wuhan AI Research（武汉人工智能研究院）
Beyond Single-Granularity Prompts: A Multi-Scale Chain-of-Thought Prompt Learning for Graph
School of Computer Science and Technology（计算机科学与技术学院）；Xidian University（西安电子科技大学）；School of Artificial Intelligence（人工智能学院）；The Hong Kong University of Science and Technology（香港科学与技术大学）
Zero-Shot Chain-of-Thought Reasoning Guided by Evolutionary Algorithms in Large Language Models
School of Intelligence Science and Technology, Peking University（北京大学智能科学与技术学院）；Institute for Artificial Intelligence, Peking University（北京大学人工智能研究院）；National Key Laboratory of General Artificial Intelligence（通用人工智能国家重点实验室）；Key Laboratory of Machine Perceptron (MOE), Peking University（北京大学机器感知教育部重点实验室）
