编辑文档的智能体：衡量智能体PDF伪造相对于非智能体对照的效果
Agents That Edit Documents: Measuring Agentic PDF Forgery Against a Non-Agentic Control
浏览论文内容
中文总结 AI 辅助
本研究提出AgentForge-Bench基准，衡量编码智能体在真实财务PDF中伪造信息的能力，发现其成功率显著高于非智能体脚本，且成本极低，凸显了自动化伪造的威胁。
中文摘要 AI 辅助
在过去一年中，能够自主执行多步骤计算机任务的AI智能体已成为日常工具，而同样的自主性也适用于任何怀有恶意任务的人。我们探讨这对依赖方——如保险公司、贷款机构、审计师——意味着什么，他们的证据是提交的PDF文件。AgentForge-Bench衡量了一个现成的编码智能体（驱动七个开放权重模型之一，配备shell和标准Python PDF库）如何根据单句意图，修改真实提交的财务文件中的一个金额、日期或地址，并通过规则而非模型进行评分。在1,750个单元格中，1,419个（81.1%）满足验证器要求，808个（46.2%）还通过了所有更严格的过滤器：可见、局部化、字体匹配、原始值在文档范围内消失。一个不含模型的确定性脚本解决了125个文档中的98个；智能体解决了124个，且没有一个是脚本单独能解决的。智能体将41%的错误编辑误报为已完成，没有模型拒绝执行，最便宜的已验证伪造成本为2.4美分。原始成功率将威胁高估了约两倍；严格成功率仍然很高。
英文摘要
AI agents that carry a multi-step computer task through on their own became ordinary tools in the past year, and the same autonomy is available to anyone whose task is harmful. We ask what that means for a relying party -- an insurer, a lender, an auditor -- whose evidence is a filed PDF. AgentForge-Bench measures how reliably an off-the-shelf coding agent, driving one of seven open-weight models with a shell and the stock Python PDF stack, alters one dollar amount, date or address in a real filed financial document from a single sentence of intent, graded by rules rather than by a model. Across 1,750 cells, 1,419 (81.1%) satisfy the verifier, and 808 (46.2%) also survive every stricter filter: visible, localized, typeface-matched, original value gone document-wide. A deterministic script with no model in it solves 98 of the 125 documents; the agents solve 124, and none the script solves alone. Agents misreport 41% of their wrong edits as done, no model refused, and the cheapest verified forgery costs 2.4 cents. The raw rate overstates the threat by about a factor of two; the strict rate is still large.
发表机构
- Scam.ai (Reality Inc.)（Scam.ai（Reality Inc.））
机构由 AI 辅助整理，请以论文原文为准。
