通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。通知：联邦节假日调休：哥伦布日 / 原住民日（Columbus Day / Indigenous Peoples' Day），今日（13日）无更新。
arXiv 2506.19143cs.LGcs.AIcs.CL
思想锚点：哪些大语言模型推理步骤至关重要？
Thought Anchors: Which LLM Reasoning Steps Matter?
MATS
Paul C. Bogdan, Uzay Macar, Neel Nanda, Arthur Conmy
备注：
Paul C. Bogdan and Uzay Macar contributed equally to this work, and their listed order was determined by coinflip. Neel Nanda and Arthur Conmy contributed equally to this work as senior authors, and their listed order was determined by coinflip
Current frontier large-language models rely on reasoning to achieve state-of-the-art performance. Many existing interpretability are limited in this area, as standard methods have been designed to study single forward passes of a model rather than the multi-token computational steps that unfold during reasoning. We argue that analyzing reasoning traces at the sentence level is a promising approach to understanding reasoning processes. We introduce a black-box method that measures each sentence's counterfactual importance by repeatedly sampling replacement sentences from the model, filtering for semantically different ones, and continuing the chain of thought from that point onwards to quantify the sentence's impact on the distribution of final answers. We discover that certain sentences can have an outsized impact on the trajectory of the reasoning trace and final answer. We term these sentences \textit{thought anchors}. These are generally planning or uncertainty management sentences, and specialized attention heads consistently attend from subsequent sentences to thought anchors. We further show that examining sentence-sentence causal links within a reasoning trace gives insight into a model's behavior. Such information can be used to predict a problem's difficulty and the extent different question domains involve sequential or diffuse reasoning. As a proof-of-concept, we demonstrate that our techniques together provide a practical toolkit for analyzing reasoning models by conducting a detailed case study of how the model solves a difficult math problem, finding that our techniques yield a consistent picture of the reasoning trace's structure. We provide an open-source tool (thought-anchors.com) for visualizing the outputs of our methods on further problems. The convergence across our methods shows the potential of sentence-level analysis for a deeper understanding of reasoning models.
Think Consistently, Reason Efficiently: Energy-Based Calibration for Implicit Chain-of-Thought
University of Oxford（牛津大学）；Tsinghua University（清华大学）；Tencent（腾讯）；Southern University of Science and Technology（南方科技大学）；National University of Singapore（新加坡国立大学）
We explore how generating a chain of thought -- a series of intermediate reasoning steps -- significantly improves the ability of large language models to perform complex reasoning. In particular, we show how such reasoning abilities emerge naturally in sufficiently large language models via a simple method called chain of thought prompting, where a few chain of thought demonstrations are provided as exemplars in prompting. Experiments on three large language models show that chain of thought prompting improves performance on a range of arithmetic, commonsense, and symbolic reasoning tasks. Th…
A Verifier-Guided Explainable Reasoning Framework with Gold-Anchored QLoRA, Task-Aware Mixture-of-Experts, and Group-Relative RLVR
University of Information Technology (UIT)（信息科技大学）；Ho Chi Minh City University of Technology (HCMUT)（胡志明市理工大学）；Vietnam National University, Ho Chi Minh City（越南国家大学胡志明市分校）；University of Economics Ho Chi Minh City (UEH)（胡志明市经济大学）
Chain-of-thought prompting combined with pre-trained large language models has achieved encouraging results on complex reasoning tasks. In this paper, we propose a new decoding strategy, self-consistency, to replace the naive greedy decoding used in chain-of-thought prompting. It first samples a diverse set of reasoning paths instead of only taking the greedy one, and then selects the most consistent answer by marginalizing out the sampled reasoning paths. Self-consistency leverages the intuition that a complex reasoning problem typically admits multiple different ways of thinking leading to i…
Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件工程学院）；Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；Shanghai Artificial Intelligence Laboratory（上海人工智能实验室）；Harbin Institute of Technology, Shenzhen（深圳哈尔滨工业大学）；University of Science and Technology Beijing（北京科技大学）
ThoughtSource: A central hub for large language model reasoning data
Medical University of Vienna（维也纳医科大学）；Technical University of Denmark（丹麦技术大学）；Copenhagen University Hospital（哥本哈根大学医院）；The Royal Institute of Technology (KTH)（皇家理工学院（KTH））
Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learning
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou)（香港科学与技术大学（广州）人工智能方向）；Didichuxing Co. Ltd（滴滴出行有限公司）；CSE, The Hong Kong University of Science and Technology（香港科学与技术大学计算机科学与工程系）
Answer-Centric or Reasoning-Driven? Uncovering the Latent Memory Anchor in LLMs
Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所）；University of Chinese Academy of Sciences（中国科学院大学）；UC Merced（加州大学默塞德分校）；The University of Queensland（昆士兰大学）；Dalian Minzu University（大连民族大学）
