The increasing popularity of \emph{reasoning} models -- language models that output a series of reasoning or thought tokens before producing an answer -- is justified, in part, by theoretical results showing that chain-of-thought (CoT) transformers can simulate Turing machines, and thus perform arbitrary computation. However, the Turing machine, while suitable for complexity-theoretic analysis, is not convenient, intuitive, or efficient for discussing algorithms. Algorithms are typically designed and analyzed at a higher level of abstraction, captured by the \emph{Word RAM} model with random-access memory and unit-cost operations on $\bigO(\log n)$-bit words. As a result, Word RAM algorithms can be substantially more efficient than their Turing machine counterparts, raising the question: \emph{Can CoT transformers efficiently simulate Word RAM algorithms?} For instance, can they sort $n$ items in $\bigO(n \log n)$ steps or run Dijkstra's algorithm in $\bigO(E + V \log V)$ steps? We answer affirmatively, up to poly-logarithmic overhead. We first establish this for finite-precision transformers with poly-logarithmic width and rightmost unique hard attention, then strengthen the result to two more practical settings with finite width and log-precision: \emph{continuous} CoT, where reasoning takes the form of vectors rather than tokens, and a \emph{hybrid} architecture in which transformer layers sit atop a recurrent (linear RNN) layer. In all three cases, we find that CoT \emph{can} efficiently simulate any Word RAM algorithm with only a poly-logarithmic overhead in $n$. This overhead reduces to log-square when the Word RAM has a ``flat'' instruction set, and only logarithmic for multiplication-free flat instructions -- in stark contrast to known CoT simulations of Turing machines, which require quadratic overhead over Word RAM.
Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought
University of Oxford（牛津大学）；Tsinghua University（清华大学）；Tencent（腾讯）；Southern University of Science and Technology（南方科技大学）；National University of Singapore（新加坡国立大学）
A*-Thought: Efficient Reasoning via Bidirectional Compression for Low-Resource Settings
Beijing University of Posts and Telecommunications（北京邮电大学）；Dept. of Comp. Sci. & Tech., Tsinghua University（清华大学计算机科学与技术系）；Northeastern University（东北大学）；Institute for AI, Tsinghua University（清华大学人工智能研究院）；Beijing National Research Center for Information Science and Technology（北京信息科学研究中心）
Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learning
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）人工智能方向）；Didichuxing Co. Ltd（滴滴出行有限公司）；CSE, The Hong Kong University of Science and Technology（香港科学与技术大学计算机科学与工程系）
OCEAN: Offline Chain-of-thought Evaluation and Alignment in Large Language Models
UC San Diego（加州大学圣迭戈分校）；The University of New South Wales（新南威尔士大学）；East China Normal University（华东师范大学）；Adobe Research（奥多比研究中心）；CSIRO’s Data61（联邦科学与工业研究组织数据61部门）
Beyond Single-Granularity Prompts: A Multi-Scale Chain-of-Thought Prompt Learning for Graph
School of Computer Science and Technology（计算机科学与技术学院）；Xidian University（西安电子科技大学）；School of Artificial Intelligence（人工智能学院）；The Hong Kong University of Science and Technology（香港科学与技术大学）
Expanding Reasoning Potential in Foundation Model by Learning Diverse Chains of Thought Patterns
School of Computer Science, Peking University（北京大学计算机科学系）；Meituan（美团）；State Key Laboratory of Multimedia Information Processing, Peking University（北京大学多媒体信息处理国家重点实验室）
Zhejiang University School of Medicine（浙江大学医学院）；First Affiliated Hospital, Zhejiang University School of Medicine（浙江大学医学院附属第一医院）；Liangzhu Laboratory（良渚实验室）；Zhejiang University（浙江大学）；Nantong University（南通大学）；Hunan First Normal University（湖南第一师范学院）
Due to the lack of a method to efficiently represent the multimodal information of a protein, including its structure and sequence information, predicting compound-protein binding affinity (CPA) still suffers from low accuracy when applying machine learning methods. To overcome this limitation, in a novel end-to-end architecture (named FeatNN), we develop a coevolutionary strategy to jointly represent the structure and sequence features of proteins and ultimately optimize the mathematical models for predicting CPA. Furthermore, from the perspective of data-driven approach, we proposed a ration…
This paper establishes theoretical bonafides for implicit concurrent multivariate effect evaluation--implicit concurrency for short---a broad and versatile computational learning efficiency thought to underlie general-purpose, non-local, noise-tolerant optimization in genetic algorithms with uniform crossover (UGAs). We demonstrate that implicit concurrency is indeed a form of efficient learning by showing that it can be used to obtain close-to-optimal bounds on the time and queries required to approximately correctly solve a constrained version (k=7, η=1/5) of a recognizable computational lea…
