LDM$^2$:一种具备动态记忆增强、模仿人类认知的大型决策模型
LDM$^2$: A Large Decision Model Imitating Human Cognition with Dynamic Memory Enhancement
AI总结:
针对大语言模型决策依赖人工提示、泛化性差的问题,提出具备动态记忆机制的LDM$^2$模型,通过记忆形成与树状探索优化的两阶段流程提升决策效果,实验验证其在得分与成功率上优于基线。
摘要:
随着大语言模型(LLMs)的快速发展,人们迫切期望能借助LLMs进行决策以实现通用人工智能。现有大多数方法利用人工构建的示例提示LLMs模仿人类的决策过程,但设计最优提示难度较大,且模式化的提示难以推广到更复杂的环境中。本文提出了一种名为带记忆的大型决策模型(Large Decision Model with Memory,LDM$^2$)的新型模型,该模型利用动态记忆机制构建动态提示,引导LLMs根据当前面临的状态做出恰当决策。LDM$^2$包含两个阶段:记忆形成阶段与记忆优化阶段。在记忆形成阶段,模型利用LLMs强大的总结能力,将人类行为分解为状态-动作元组;随后将这些元组存储在记忆库中,记忆的索引由LLMs生成,以便根据当前状态检索最相关的已存储元组子集。在记忆优化阶段,LDM$^2$采用树状探索发现更合适的决策过程,并通过添加有价值的状态-动作元组丰富记忆库。探索与记忆增强的动态循环让LDM$^2$对全局环境有了更深入的理解。在两种交互环境中开展的大量实验表明,LDM$^2$在得分和成功率两方面均优于各基线模型,证明了其有效性。
英文摘要:
With the rapid development of large language models (LLMs), it is highly demanded that LLMs can be adopted to make decisions to enable the artificial general intelligence. Most approaches leverage manually crafted examples to prompt the LLMs to imitate the decision process of human. However, designing optimal prompts is difficult and the patterned prompts can hardly be generalized to more complex environments. In this paper, we propose a novel model named Large Decision Model with Memory (LDM$^2$), which leverages a dynamic memory mechanism to construct dynamic prompts, guiding the LLMs in making proper decisions according to the faced state. LDM$^2$ consists of two stages: memory formation and memory refinement. In the former stage, human behaviors are decomposed into state-action tuples utilizing the powerful summarizing ability of LLMs. Then, these tuples are stored in the memory, whose indices are generated by the LLMs, to facilitate the retrieval of the most relevant subset of memorized tuples based on the current state. In the latter stage, our LDM$^2$ employs tree exploration to discover more suitable decision processes and enrich the memory by adding valuable state-action tuples. The dynamic circle of exploration and memory enhancement provides LDM$^2$ a better understanding of the global environment. Extensive experiments conducted in two interactive environments have shown that our LDM$^2$ outperforms the baselines in terms of both score and success rate, which demonstrates its effectiveness.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。