briefly, Then Decide: 通过累积熵调节缓解LLM过度思考
Explore Briefly, Then Decide: Mitigating LLM Overthinking via Cumulative Entropy Regulation
AI总结：
本文提出TECA指标和
摘要：
大型语言模型（LLMs）在复杂问题上展示了通过长链思维（CoT）推理的出色推理能力。然而，它们常常表现出过度思考的问题，即为简单问题生成不必要的长推理步骤。为了解决这一问题，我们引入了一个新的度量指标Token Entropy Cumulative Average（TECA），用于衡量推理过程中探索的广度。我们进一步提出了一种新的推理范式，名为
英文摘要：
Large Language Models (LLMs) have demonstrated remarkable reasoning abilities on complex problems using long Chain-of-Thought (CoT) reasoning. However, they often suffer from overthinking, meaning generating unnecessarily lengthy reasoning steps for simpler problems. This issue may degrade the efficiency of the models and make them difficult to adapt the reasoning depth to the complexity of problems. To address this, we introduce a novel metric Token Entropy Cumulative Average (TECA), which measures the extent of exploration throughout the reasoning process. We further propose a novel reasoning paradigm named "Explore Briefly, Then Decide", with an associated Cumulative Entropy Regulation (CER) mechanism. This paradigm leverages TECA to help the model dynamically determine the optimal point to conclude its thought process and provide a final answer, thus achieving efficient reasoning. Experimental results across diverse mathematical benchmarks show that our approach substantially mitigates overthinking without sacrificing problem solving ability. With our thinking paradigm, the average response length decreases by up to 71% on simpler datasets, demonstrating the effectiveness of our method in creating a more efficient and adaptive reasoning process.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
