arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2508.05282cs.CL

并非所有错误都是同等重要:ASCoT解决高效大语言模型推理中的晚期脆弱性

Not All Errors Are Created Equal: ASCoT Addresses Late-Stage Fragility in Efficient LLM Reasoning

Xi’an Jiaotong University(西安交通大学);Peking University(北京大学);Institute of Automation, CASIA(中国科学院自动化研究所);Bytedance(字节跳动)

Dongxu Zhang, Yujun Wu, Yiding Sun, Jinnan Yang, Ning Yang, Jihua Zhu, Miao Xin, Baoliang Tian

AI总结:

ASCoT通过自适应校正机制提升大语言模型推理效率与可靠性,减少计算资源消耗的同时保持高准确性。

摘要:

尽管链式思维(CoT)提示方法增强了大型语言模型(LLMs)的能力,但确保推理可靠性仍是一个开放性挑战。与主流的级联故障假说(认为早期错误最为有害)相反,我们发现一种反直觉现象,称为晚期脆弱性:在后期推理阶段引入的错误更可能破坏最终答案。为此,我们引入了ASCoT(自适应自我校正链式思维),一种在效率与鲁棒验证之间取得平衡的方法。ASCoT首先使用语义剪枝来压缩冗余步骤,然后利用自适应验证管理器(AVM)通过位置影响分数优先处理高风险的后期步骤,仅在必要时触发多视角自我校正引擎(MSCE)。在GSM8K和MATH-500上的实验表明,ASCoT有效重新分配计算资源:它在LLaMA-3.1-8B上减少了21%-30%的token使用量,同时准确性下降非常小(<1.8%),实现了推理效率与推理忠实度之间的更优权衡。

英文摘要:

While Chain-of-Thought (CoT) prompting empowers Large Language Models (LLMs), ensuring reasoning reliability remains an open challenge. Contrary to the prevailing cascading failure hypothesis which posits that early errors are most detrimental, we identify a counter-intuitive phenomenon termed \textbf{Late-Stage Fragility}: errors introduced in later reasoning stages are significantly more prone to corrupting final answers. To address this, we introduce ASCoT (Adaptive Self-Correction Chain-of-Thought), a method harmonizing efficiency with robust verification. ASCoT first employs semantic pruning to compress redundant steps, then utilizes an Adaptive Verification Manager (AVM) to prioritize high risk, late-stage steps via a positional impact score, triggering a Multi-Perspective Self-Correction Engine (MSCE) only when necessary. Experiments on GSM8K and MATH-500 demonstrate that ASCoT effectively reallocates computational resources: it reduces token usage by 21\%--30\% for LLaMA-3.1-8B with negligible accuracy drops ($<1.8\%$), achieving a superior trade-off between inference efficiency and reasoning fidelity.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑