共享智能体记忆中认知接纳的基准与诊断研究
A Benchmark and Diagnostic Study of Epistemic Admission in Shared Agent Memory
浏览论文内容
中文总结 AI 辅助
针对共享智能体记忆中的声明接纳问题，提出相关提升基准（CPB），评估八种接纳策略，发现来源去重误拒真声明、来源类型门控降低虚假采纳，且无策略能一致拒绝虚假声明。
中文摘要 AI 辅助
评估共享智能体记忆中的声明接纳具有挑战性，因为重复声明可能被误认为是独立证据。智能体可能复制或改写检索到的信念，而接纳虚假声明会使后续智能体暴露于该虚假信息。为研究此问题，我们引入了相关提升基准（Correlated Promotion Benchmark, CPB），该基准评估候选声明是否应被接纳到共享存储中。CPB-Static从公开标注来源构建冻结的测试分割，并带有固定的黄金动作。CPB-Live在共享存储上运行多智能体团队，记录所有写入和检索操作，并追踪由每个场景定义的来源谱系。一个独立的消费者仅从存储中作答。我们评估了四个智能体家族中的八种接纳策略。结果表明，对来源进行去重的策略在拒绝虚假声明的同时也拒绝了许多真实声明，而保留答案覆盖范围的策略接纳的虚假声明数量几乎与无限制共享相当。根据声明的来源类型进行门控可将虚假采纳率降至0.06至0.09，而其他作答策略的虚假采纳率为0.22至0.47。一旦无争议的虚假信念进入记忆，消费者在所有家族的探针中会在0.97至0.99的情况下断言该信念。没有任何非预言机策略能在逐字复制、改写和声明为权威的改写中一致地拒绝虚假声明。这些发现揭示了在无法访问来源谱系的情况下接纳策略的局限性。
英文摘要
Evaluating claim admission in shared agent memory is challenging because repeated claims may be mistaken for independent evidence. An agent may copy or paraphrase a retrieved belief, while admitting a false claim exposes subsequent agents to it. To study this problem, we introduce the Correlated Promotion Benchmark (CPB), which evaluates whether candidate claims should be admitted to shared memory.CPB-Static constructs a frozen test split from publicly annotated sources with fixed gold actions. CPB-Live runs multi-agent teams over a shared store, records all writes and retrievals, and tracks source lineage defined by each scenario. A separate consumer answers from the store alone. We evaluate eight admission policies across four agent families. Our results show that policies which deduplicate sources reject many true claims alongside false ones, whereas policies preserving answer coverage admit nearly as many false claims as unrestricted sharing. Gating on declared source type reduces false adoption to 0.06--0.09, compared with 0.22--0.47 for other answering policies. Once an uncontested false belief enters memory, the consumer asserts it in 0.97--0.99 of probes across all families. No non-oracle policy consistently rejects false claims across verbatim copies, paraphrases, and paraphrases declared authoritative. These findings reveal the limitations of admission policies without access to source lineage.
发表机构
- University of Southern Queensland（南昆士兰大学）
- University of New South Wales（新南威尔士大学）
- Aikaier（艾凯尔）
- Nanjing University（南京大学）
- Facebook（脸书）
机构由 AI 辅助整理，请以论文原文为准。
