arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2411.09111cs.LG
收藏

降低推理成本:通过稀疏注意力机制优化思维链的路径

Reducing Reasoning Costs: The Path of Optimization for Chain of Thought via Sparse Attention Mechanism

Nicolaus Copernicus University(哥白尼大学);UCSI University(思特雅大学)

Libo Wang

AI总结:

为降低大模型思维链推理成本,本研究提出稀疏注意力机制并构建GiantRabbit模型,实验证明其在MIT线性代数测试中推理时间与思维链长度显著优于o1 Preview。

摘要:

为了解决大语言模型中思维链导致的推理成本激增问题,本研究提出使用一种仅关注少数相关 token 的稀疏注意力机制。研究者构建了一种新的注意力机制,并使用基于自定义 GPT 训练的 GiantRabbit 作为实验工具。实验测试并比较了该模型与 o1 Preview 在解决 MIT OpenCourseWare 线性代数测试题时的推理时间、正确性得分和思维链长度。结果表明,GiantRabbit 的推理时间与思维链长度显著低于 o1 Preview。这验证了稀疏注意力机制优化思维链推理的可行性。详细的架构细节和实验过程已上传至 Github,链接为:https://github.com/brucewang123456789/GeniusTrail.git。

英文摘要:

In order to address the chain of thought in the large language model inference cost surge, this research proposes to use a sparse attention mechanism that only focuses on a few relevant tokens. The researcher constructed a new attention mechanism and used GiantRabbit trained with custom GPTs as an experimental tool. The experiment tested and compared the reasoning time, correctness score and chain of thought length of this model and o1 Preview in solving the linear algebra test questions of MIT OpenCourseWare. The results show that GiantRabbit's reasoning time and chain of thought length are significantly lower than o1 Preview. It verifies the feasibility of sparse attention mechanism for optimizing chain of thought reasoning. Detailed architectural details and experimental process have been uploaded to Github, the link is:https://github.com/brucewang123456789/GeniusTrail.git.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑