arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2501.05752cs.AIcs.CL
收藏

面向语言模型高效问题求解的自适应门控语义探索

Semantic Exploration with Adaptive Gating for Efficient Problem Solving with Language Models

Department of Computer Science and Engineering, POSTECH, South Korea(计算机科学与工程系,POSTECH,韩国);Graduate School of Artificial Intelligence, POSTECH, South Korea(人工智能研究生院,POSTECH,韩国)

Sungjae Lee, Hyejin Park, Jaechang Kim, Jungseul Ok

AI总结:

提出 SEAG 框架,通过自适应门控按需触发树搜索并合并语义等价推理步骤,在 GSM8K、ARC 等基准上平均提升准确率 4.3%,计算成本仅为现有方法的 31%。

摘要:

近期,大型语言模型(LLM)的进展在各类需要多步推理方法的复杂任务中展现出显著潜力,例如通过树搜索探索多样推理路径。然而,现有方法通常存在计算低效和冗余问题。首先,它们忽视任务难度的多样性,导致即使是简单任务也进行不必要的大规模搜索。其次,它们忽略推理路径的语义,造成对语义相同路径的冗余探索。为解决这些局限,我们提出 Semantic Exploration with Adaptive Gating(SEAG),这是一种计算高效的方法。SEAG采用自适应门控机制,根据前置简单推理方法给出答案的置信度,动态决定是否执行树搜索。此外,其基于树的探索会合并语义相同的推理步骤,在保持甚至提高准确率的同时减少冗余探索。大量实验表明,在包括 GSM8K 和 ARC 在内的复杂推理基准上,并使用 Llama2、Llama3 和 Mistral 等多种语言模型时,与现有基于树搜索的方法相比,SEAG 平均将准确率显著提高 4.3%,而计算成本仅为其 31%。代码可在 https://github.com/ml-postech/SEAG-semantic-exploration-with-adaptive-gating 获取。

英文摘要:

Recent advancements in large language models (LLMs) have shown remarkable potential in various complex tasks requiring multi-step reasoning methods like tree search to explore diverse reasoning paths. However, existing methods often suffer from computational inefficiency and redundancy. First, they overlook the diversity of task difficulties, leading to unnecessarily extensive searches even for easy tasks. Second, they neglect the semantics of reasoning paths, resulting in redundant exploration of semantically identical paths. To address these limitations, we propose Semantic Exploration with Adaptive Gating (SEAG), a computationally efficient method. SEAG employs an adaptive gating mechanism that dynamically decides whether to conduct a tree search, based on the confidence level of answers from a preceding simple reasoning method. Furthermore, its tree-based exploration consolidates semantically identical reasoning steps, reducing redundant explorations while maintaining or even improving accuracy. Our extensive experiments demonstrate that SEAG significantly improves accuracy by 4.3% on average while requiring only 31% of computational costs compared to existing tree search-based methods on complex reasoning benchmarks including GSM8K and ARC with diverse language models such as Llama2, Llama3, and Mistral. Our code is available at https://github.com/ml-postech/SEAG-semantic-exploration-with-adaptive-gating .

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑