使用Certaindex高效扩展LLM推理
Efficiently Scaling LLM Reasoning with Certaindex
AI总结：
本文提出算法无关的指标Certaindex来量化测试时推理的答案稳定化现象，通过提前退出和动态分配等机制，在Dynasor系统中实现50%计算节省和3.3倍吞吐量提升且不损失准确率。
摘要：
思维链、自洽性和MCTS等测试时推理算法能增强LLM的问题解决能力，但可能浪费地生成大量token而无法提高准确率。同时，我们观察到这些算法表现出答案稳定化：其中间解通常在某个点后停止变化，进一步投入计算也不会改变最终答案。为量化此现象，我们引入Certaindex，一种与算法无关的指标，用于测量这种演化稳定性，指示何时进一步计算不太可能改变最终结果。Certaindex是轻量级的，可通过提前退出加速推理程序推理，并进一步实现动态token分配、组调度，以及在与实际LLM服务系统集成时的诸多机会。为量化实际收益，我们将Certaindex作为调度器构建到Dynasor（我们的推理感知LLM服务系统）中，在实际工作负载中展示了高达50%的计算节省和3.3倍的吞吐量提升，且准确率无下降。代码可在https://github.com/hao-ai-lab/Dynasor.git获取。
英文摘要：
Test-time reasoning algorithms such as chain-of-thought, self-consistency, and MCTS enhance LLM problem-solving but can wastefully generate many tokens without improving accuracy. At the same time, we observe that these algorithms exhibit answer stabilization: their intermediate solutions often cease to change after a certain point, and further investment of compute does not change their final answer. To quantify this phenomenon, we introduce Certaindex, an algorithm-agnostic metric measuring this evolving stability, signaling when further computation is unlikely to alter the final result. Certaindex is lightweight, can accelerate reasoning program inference via early exit, and further enables dynamic token allocation, gang scheduling, and many opportunities when integrated with real-world LLM serving systems. To quantify real-world benefits, we built Certaindex as a scheduler into Dynasor, our reasoning-aware LLM serving system, and demonstrate up to 50% compute savings and 3.3x higher throughput in real workloads with no accuracy drop. Our code is available at https://github.com/hao-ai-lab/Dynasor.git
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
