CreDes:利用大语言模型解决长程推理问题的因果推理增强与双端搜索
CreDes: Causal Reasoning Enhancement and Dual-End Searching for Solving Long-Range Reasoning Problems using LLMs
AI总结:
本文提出CreDes框架,通过因果关系增强保证推理与状态转移一致,并以双端搜索缩小长程搜索空间,从而提升LLM组合优化任务的准确率与效率。
摘要:
大语言模型(LLM)在处理涉及长程推理的组合优化问题时表现出局限性,部分原因在于因果幻觉和巨大的搜索空间。针对因果幻觉,即推理与相应状态转移之间的不一致,本文提出因果关系增强(CRE)机制,该机制结合因果干预和个体处理效应(ITE),以确保推理每一步与状态转移之间具备坚实的因果正确性。针对限制现有以单向搜索为特征的模型性能的长因果链和巨大搜索空间,本文提出一种双端搜索(DES)方法,通过在因果概率树上同时从初始状态和目标状态出发来寻找解决方案。通过集成CRE和DES(CreDes),本文模型实现了同步多步推理,避免了像思维链(CoT)那样级联多个单步推理所带来的低效。实验表明,在长程推理任务中,CreDes在准确率和时间效率两方面均显著优于现有的最先进(SOTA)解决方案。
英文摘要:
Large language models (LLMs) have demonstrated limitations in handling combinatorial optimization problems involving long-range reasoning, partially due to causal hallucinations and huge search space. As for causal hallucinations, i.e., the inconsistency between reasoning and corresponding state transition, this paper introduces the Causal Relationship Enhancement (CRE) mechanism combining cause-effect interventions and the Individual Treatment Effect (ITE) to guarantee the solid causal rightness between each step of reasoning and state transition. As for the long causal range and huge search space limiting the performances of existing models featuring single-direction search, a Dual-End Searching (DES) approach is proposed to seek solutions by simultaneously starting from both the initial and goal states on the causal probability tree. By integrating CRE and DES (CreDes), our model has realized simultaneous multi-step reasoning, circumventing the inefficiencies from cascading multiple one-step reasoning like the Chain-of-Thought (CoT). Experiments demonstrate that CreDes significantly outperforms existing State-Of-The-Art (SOTA) solutions in long-range reasoning tasks in terms of both accuracy and time efficiency.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。