arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.06124cs.CLcs.AI
收藏

利用 Hierarchical Aggregate Tree(层次聚合树)增强检索增强生成的长期记忆

Enhancing Long-Term Memory using Hierarchical Aggregate Tree for Retrieval Augmented Generation

Amrita School of Artificial Intelligence(阿姆利塔人工智能学院);Amrita Vishwa Vidyapeetham(阿姆利塔维什瓦维迪亚皮塔姆大学)

Aadharsh Aadhithya A, Sachin Kumar S, Soman K. P

AI总结:

针对 LLM 长对话上下文受限问题,提出 HAT 层次聚合树记忆结构,将上下文选择建模为最优树遍历并递归聚合相关信息,实验表明其提升多轮对话连贯性与摘要质量且避免参数指数增长。

摘要:

大型语言模型的上下文容量有限,这阻碍了其在长对话上的推理。我们提出 Hierarchical Aggregate Tree(层次聚合树)记忆结构,通过条件化树遍历递归聚合相关对话上下文。HAT 封装来自子节点的信息,从而在控制深度的同时实现广泛覆盖。我们将寻找最佳上下文表述为最优树遍历问题。实验表明,与基线上下文相比,HAT 改善了对话连贯性和摘要质量,证明该技术可用于多轮推理且不会带来指数级参数增长。这种记忆增强使 LLM 能够产生更一致、更有依据的长篇对话。

英文摘要:

Large language models have limited context capacity, hindering reasoning over long conversations. We propose the Hierarchical Aggregate Tree memory structure to recursively aggregate relevant dialogue context through conditional tree traversals. HAT encapsulates information from children nodes, enabling broad coverage with depth control. We formulate finding best context as optimal tree traversal. Experiments show HAT improves dialog coherence and summary quality over baseline contexts, demonstrating the techniques effectiveness for multi turn reasoning without exponential parameter growth. This memory augmentation enables more consistent, grounded longform conversations from LLMs

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑