发表机构
Sungkyunkwan University（成均馆大学）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
本研究通过64个AgentDojo案例审计LLM代理日志取证重建，发现仅凭事实一致性不足，需保留显式记录绑定以区分有支撑发现与正确猜测。
AI 中文摘要
LLM代理行为的取证重建不仅需要恢复正确的数值，还需要确定哪个保留的记录支持该发现。工具日志、生成的解释和本地引用标识符捕获了该证据的不同部分，然而，除非引用标识符与记录的绑定被保留，否则它不能确立来源。我们使用来自保存的AgentDojo Banking执行的64个可机械检查的案例来审计这一区别。两个LLM阅读器在可见证据和标识符到记录绑定的受控变化下重建来源关系。我们分别评估完整记录一致性、基于证据的发现、有理由的弃权（不执行）和无支撑的断言。使用原始标识符且无绑定表时，Sonnet恢复了每个字面来源位置，但在28个需要缺失关系的案例中，有26个做出了无支撑的引用-来源断言；其中22个仍然与完整参考匹配。添加显式绑定改善了两个阅读器的有根据重建，而仅重命名标识符未提供一致的补救措施。一个确定性的同包比较器正确解决了有界任务或全程弃权（不执行）。这些结果表明，仅事实一致性不足以评估代理日志的取证重建，并促使保留显式记录绑定以区分有支撑的发现与正确的猜测。
英文摘要
Forensic reconstruction of LLM-agent actions requires not only recovering the correct value, but establishing which preserved record supports that finding. Tool logs, generated explanations, and local citation identifiers capture different parts of this evidence, yet a citation identifier does not establish a source unless its binding to a record is preserved. We audit this distinction using 64 mechanically checkable cases from saved AgentDojo Banking executions. Two LLM readers reconstruct source relationships under controlled variations in visible evidence and identifier-to-record bindings. We separately evaluate complete-record agreement, evidence-grounded findings, justified abstention, and unsupported assertions. With original identifiers and no binding table, Sonnet recovered every literal source location but made unsupported citation-source assertions in 26 of 28 cases requiring the missing relation; 22 nevertheless matched the complete reference. Adding explicit bindings improved grounded reconstruction for both readers, whereas identifier renaming alone provided no consistent remedy. A deterministic same-packet comparator correctly resolved the bounded task or abstained throughout. These results show that factual agreement alone is insufficient for evaluating forensic reconstruction of agent logs and motivate preserving explicit record bindings to distinguish supported findings from correct guesses.
Comments11 pages, 2 figures
