LLMs是单线程推理者：揭秘软思考的工作机制
LLMs are Single-threaded Reasoners: Demystifying the Working Mechanism of Soft Thinking
AI总结：
本文通过探测技术揭示LLMs在软思考中表现为单线程推理者，受贪婪陷阱限制，并提出随机软思考（结合Gumbel-Softmax）以突破该限制，在八个推理基准上取得更优性能。
摘要：
人类认知自然涉及抽象和流动的概念，而现有的推理模型通常依赖生成离散令牌，这可能限制其表达能力。最近的进展旨在通过使大型语言模型（LLMs）生成软性、抽象的令牌来解决这一限制，从而在连续概念空间内促进推理。在本文中，我们通过一套探测技术系统分析各种LLMs的内部行为，研究其软思考能力。与普遍认为软思考支持并行探索多种推理路径的观点相反，我们的发现揭示LLMs表现为单线程推理者——它们主要依赖软输入中概率最高的令牌来预测下一步。这种行为引发了一个贪婪反馈循环，抑制了替代推理路径，并削弱了通过软令牌传递更丰富信息的益处。为了解决这一贪婪陷阱，我们提出了随机软思考，通过引入随机性来摆脱这一贪婪陷阱。我们的实验表明，加入随机性——特别是使用Gumbel-Softmax技巧——可以缓解朴素方法的局限性并释放软思考的潜力，在八个推理基准上取得更优性能。我们进一步证明，随机软思考相比传统COT展现出更强的探索潜力。我们的发现加深了对连续推理的理解，并为未来利用强化学习改进软思考的工作奠定了基础。
英文摘要：
Human cognition naturally engages with abstract and fluid concepts, whereas existing reasoning models often rely on generating discrete tokens, potentially constraining their expressive capabilities. Recent advancements aim to address this limitation by enabling large language models (LLMs) to generate soft, abstract tokens, thus facilitating reasoning within a continuous concept space. In this paper, we investigate the Soft Thinking capabilities of various LLMs through a systematic analysis of their internal behavior using a suite of probing techniques. Contrary to the prevailing belief that Soft Thinking supports parallel exploration of diverse reasoning paths, our findings reveal that LLMs behave as single-threaded reasoners--they predominantly rely on the token with the highest probability in the soft input to predict the next step. This behavior induces a greedy feedback loop that suppresses alternative reasoning paths and undermines the benefits of transmitting richer information via Soft Tokens. To address this Greedy Pitfall, we propose Stochastic Soft Thinking, which introduces stochasticity to break free from this Greedy Pitfall. Our experiments demonstrate that incorporating randomness--particularly with the Gumbel-Softmax trick--can alleviate the limitations of vanilla approaches and unleash the potential of Soft Thinking, resulting in superior performance across eight reasoning benchmarks. We further demonstrate that Stochastic Soft Thinking exhibits stronger exploration potential compared to conventional COT. Our findings deepen the understanding of continuous reasoning and establish the foundation for future work on improving Soft Thinking with Reinforcement Learning.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
