ReplayLens：审计智能体对结果的使用
ReplayLens: Auditing Agents' Use of Outcomes
浏览论文内容
中文总结 AI 辅助
ReplayLens通过四种黑盒干预分别审计记忆中的分值、名称、位置等关系，揭示智能体决策变化的具体来源，确保日志经验可安全重用。
中文摘要 AI 辅助
当智能体重用记录的经验时，决策的改变可能源于记录的分值、动作的名称或记录在存储中的位置。标准的记忆评估无法揭示是哪种关系驱动了这种改变。我们引入了ReplayLens，一种黑盒审计方法，它每次只改变存储历史中的一种关系，保持其余接口不变，并测量由此产生的决策变化。四种干预措施针对四种关系。结果重分配交换了哪些分值属于哪些动作。配对迁移将完整的动作-分值对移动到新的记录槽位。一致性重命名在历史和菜单中同时重新标记动作。关键槽位重分配同时改变分值附着和位置。一个建设性的分离实验说明了为何需要该审计：两个具有相同端点准确率的记忆写入器对相同的重放表现出不同的响应，因此传统评估无法解析潜在的依赖关系。在黑盒LLM接口上，交换分值会改变决策，而移动完整的配对则不会，从而将分值附着与记录顺序区分开来。一项有界记忆研究暴露了端点比较所遗漏的摄入顺序敏感性。在顺序实验规划中，被篡改的历史分值会重定向探索方向，并在有新鲜测量的情况下降低最终效用。一个带有密封隐藏测试的代码调试智能体在模型选择之外也表现出相同的模式。ReplayLens提供了一种关系级别的审计，用于决定记录的经验是否可以安全地合并、重排序或重新索引。
英文摘要
When an agent reuses logged experience, a changed decision may reflect the recorded score, the action's name, or the record's position in storage. Standard memory evaluations do not reveal which relationship drives that change. We introduce ReplayLens, a black-box audit that changes one relationship in the stored history at a time, holds the remaining interface fixed, and measures the resulting decision. Four interventions target four relationships. Outcome reassignment swaps which scores belong to which actions. Pair transport moves intact action-score pairs to new record slots. Consistent renaming relabels actions in both history and menu. Key-slot reassignment changes both score attachment and position. A constructive separation shows why the audit is needed: two memory writers with identical endpoint accuracy respond differently to the same replay, so conventional evaluation cannot resolve the underlying dependence. On black-box LLM interfaces, swapping scores changes decisions while moving intact pairs does not, separating score attachment from record order. A bounded-memory study exposes ingestion-order sensitivity that endpoint comparison misses. In sequential experiment planning, altered historical scores redirect exploration and reduce final utility despite fresh measurements. A code-debugging agent with sealed hidden tests shows the same pattern outside model selection. ReplayLens provides a relationship-level audit for deciding whether logged experience can be merged, reordered, or reindexed safely.
发表机构
- Shenzhen University（深圳大学）
- EasternDawn（东方黎明）
- University of Nottingham Ningbo（宁波诺丁汉大学）
机构由 AI 辅助整理，请以论文原文为准。
