arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2512.20111cs.CLcs.AIcs.LG
收藏

ABBEL: 为高效交互学习自然语言信念状态

ABBEL: Learning Natural-Language Belief States for Memory-Efficient Interaction

University of California, Berkeley(加州大学伯克利分校);Georgia Institute of Technology(佐治亚理工学院)

Aly Lidayan, Jakob Bjorner, Satvik Golechha, Kartik Goyal, Alane Suhr

AI总结:

本文提出ABBEL框架,通过显式自然语言信念状态直接监督每个摘要的信息内容,以解决传统方法在生成摘要时信息丢失或更新错误的问题,从而在保持高效内存使用的同时提升交互性能。

摘要:

随着序列决策任务的时间范围扩大,将完整交互历史保留在模型上下文中变得越来越昂贵。最近的研究通过使用递归更新的自然语言摘要来减少上下文长度,这些摘要简洁且可解释。然而,这些方法在性能上仍低于能够访问完整上下文的智能体,表明它们未能生成足够的摘要。为此,我们提出了ABBEL,一种递归摘要框架,通过显式自然语言信念状态直接监督每个摘要的信息内容。首先,我们分析了在五个领域中由前沿模型生成的信念状态,并验证了性能通常因遗漏或错误更新信息而降低。我们还发现了一些模型使用内存低效的设置,通过保留冗余信息。我们通过两种基于强化学习的方法进行微调:信念分级,通过奖励基于信息内容的信念生成来减少更新错误;峰值信念惩罚,通过鼓励压缩内存足迹最大的信念。我们证明这些方法显著缩小了与完整上下文模型的性能差距,并使ABBEL在使用67%内存的情况下,比先前的记忆智能体工作提高了40%。我们的代码可在https://github.com/jakob-bjorner/optimal-explorer-dev获取。

英文摘要:

As the time horizons of sequential decision-making tasks grow, keeping full interaction histories in model context becomes increasingly costly. Recent work reduces context lengths by instead conditioning decision-making agents on recursively updated natural-language summaries, which are concise and interpretable. However, they underperform agents with access to the full context, suggesting that they fail to generate sufficient summaries. To address this we propose ABBEL, a recursive summarization framework that isolates and directly supervises each summary's information contents in the form of explicit natural-language belief states. First, we analyze the belief states generated by frontier models under ABBEL across five domains, and verify that performance is often degraded due to omitting or incorrectly updating information. We also discover settings where models use memory inefficiently by retaining extraneous information. We target these limitations by fine-tuning with two RL-based methods: belief grading, which reduces update errors by rewarding belief generations based on their information content, and peak belief penalties, which encourage compressing the beliefs with the greatest memory footprints. We demonstrate that these methods significantly reduce the performance gap with full context models, and enable ABBEL to outperform prior memory agent work by 40% while using 67% of the memory. Our code is available at https://github.com/jakob-bjorner/optimal-explorer-dev

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑