通过基于梯度的表示优化在基础LLM中引导链式思维
Eliciting Chain-of-Thought in Base LLMs via Gradient-Based Representation Optimization
AI总结：
本文提出了一种基于概率条件生成的隐藏状态优化方法，通过梯度优化提升基础LLM的链式思维推理能力，有效解决隐藏状态操纵中的分布偏移问题。
摘要：
链式思维（CoT）推理是大型语言模型（LLMs）的关键能力，使它们能够处理复杂的多步骤任务。尽管预训练在通用文本语料库上的基础LLMs往往因缺乏专门训练而难以进行推理，但最近的研究揭示了其潜在的推理潜力与隐藏状态相关。然而，现有的隐藏状态操纵方法，如线性激活引导，由于其刚性和无约束的性质，往往导致分布偏移和文本质量下降。在本文中，我们提出了一种新颖的方法，通过基于概率条件生成的隐藏状态操纵来引导基础LLM的链式思维推理。通过将挑战重新表述为一个具有平衡似然和先验正则化框架的优化问题，我们的方法引导隐藏状态向推理导向的轨迹发展，同时保持语言的连贯性。在数学、常识和逻辑推理基准上的广泛评估表明，我们的方法在多个方面都优于现有引导方法，提供了一种理论上有根据且有效的解决方案，用于增强基础LLM的推理能力。
英文摘要：
Chain-of-Thought (CoT) reasoning is a critical capability for large language models (LLMs), enabling them to tackle com- plex multi-step tasks. While base LLMs, pre-trained on general text corpora, often struggle with reasoning due to a lack of specialized training, recent studies reveal their latent reason- ing potential tied to hidden states. However, existing hidden state manipulation methods, such as linear activation steering, suffer from limitations due to their rigid and unconstrained nature, often leading to distribution shifts and degraded text quality. In this work, we propose a novel approach for elic- iting CoT reasoning from base LLMs through hidden state manipulation grounded in probabilistic conditional generation. By reformulating the challenge as an optimization problem with a balanced likelihood and prior regularization framework, our method guides hidden states toward reasoning-oriented trajectories while preserving linguistic coherence. Extensive evaluations across mathematical, commonsense, and logical reasoning benchmarks demonstrate that our approach con- sistently outperforms existing steering methods, offering a theoretically principled and effective solution for enhancing reasoning capabilities in base LLMs.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
