是什么让智能体记忆对可靠的无答案问题处理有用?
What Makes Agent Memory Useful for Reliable Unanswerable Question Handling?
- School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院)
- Suzhou City University(苏州城市学院)
- Shanghai Key Lab of Intelligent Information Processing(上海智能信息处理重点实验室)
- College of Computer Science and Artificial Intelligence, Fudan University(复旦大学计算机科学技术与人工智能学院)
- Shanghai Innovation Institute(上海创新研究院)
机构由 AI 辅助整理,请以论文原文为准。
AI总结:
该研究在agentic RAG框架下,评估不同记忆方法对UAQ处理的作用,发现记忆提升UAQ性能具选择性,程序与规则类记忆结合互补信号效果最优,可靠UAQ记忆依赖保留可迁移行为指导。
AI中文摘要:
可靠处理无答案问题(UAQ)对于基于大语言模型(LLM)的智能体的可信赖性至关重要。尽管记忆在智能体系统中被广泛使用,但它在可靠的UAQ处理中的作用仍不明确。我们在统一的智能体检索增强生成(agentic RAG)框架下,对用于UAQ处理的智能体记忆进行了系统研究,在3个UAQ相关数据集和2个基础模型上评估了4种代表性记忆方法。我们发现,在某些设置下记忆可以提升UAQ性能,但这种提升是有选择性的而非普遍的,且在数据集偏移下仍很脆弱。有趣的是,跨模型记忆复用通常比跨数据集迁移更可行,这表明可回答性模式的偏移比基础模型本身的变化对记忆复用构成更大挑战。我们进一步发现,UAQ的提升通过决策指导比通过轨迹塑形更能得到保留,且记忆有效性强烈依赖于表征。具体而言,基于程序和规则的记忆通常为UAQ处理提供最可靠的支持,而当程序指导与互补的行为信号结合时,记忆组合最有效。总体而言,我们的发现表明,可靠的UAQ记忆更多取决于保留可迁移的行为指导,而非存储更多经验。
英文摘要:
Reliable handling of unanswerable questions (UAQs) is critical for trustworthy LLM-based agents. Although memory is widely used in agent systems, its role in reliable UAQ handling remains unclear. We present a systematic study of agent memory for UAQ handling under a unified agentic RAG framework, evaluating four representative memory methods across three UAQ-related datasets and two base models. We find that memory can improve UAQ performance in some settings, but such gains are selective rather than universal and remain fragile under dataset shift. Interestingly, cross-model memory reuse is often more feasible than cross-dataset transfer, suggesting that shifts in answerability patterns pose a greater challenge to memory reuse than changes in the base model itself. We further find that UAQ gains are more strongly preserved through decision guidance than through trajectory shaping, and that memory effectiveness depends strongly on representation. In particular, procedural and rule-based memories often provide the most reliable support for UAQ handling, while memory composition is most effective when procedural guidance is combined with complementary behavioral signals. Overall, our findings suggest that reliable UAQ memory depends less on storing larger amounts of experience and more on preserving transferable behavioral guidance.