发表机构
William & Mary; Anytime AI（威廉与玛丽学院; Anytime AI）
机构由 AI 辅助整理，请以论文原文为准。AI 中文总结
AtomCite是一个智能体框架，通过解析声明并核对页面图像来验证和纠正多页文档中的页面级引用，配合新基准DocCite，显著提升引用精确率并具备迁移性。
AI 中文摘要
大型语言模型在回答多页文档问题时，被期望引用支持性页面，然而提供的引用有时并不准确，且当前的评估在生成时或针对文本段落进行引用评分：现有基准均未评估系统能否验证和纠正已附加到答案上的页面级引用。我们提出AtomCite，一个智能体框架，它将答案解析为多个声明，将每个声明与所引用页面的图像进行核对，并应用确定性修复策略。为评估该框架，我们引入DocCite，据我们所知，这是首个针对文档图像中页面级引用验证与纠正系统的基准。基于MP-DocVQA和DUDE构建，它结合了928个经过验证的注入实例与从前沿和效率层级模型中收集的2,468个候选自然错误，其中两位标注者的审计确认了1,909个为真实错误。主要标签由确定性方式分配，而非由LLM评判员分配，人工审计作为独立的验证层。在三个模型家族（Gemini、Claude和GPT）中，AtomCite在注入基准上达到约93%的二元验证准确率，显著优于所有仅OCR的条件（包括计算匹配的对照组），并在给定相同OCR文本的情况下，超越了所有先前的基于文本的基线。其修复策略将注入混合数据上的引用精确率从构造的34%提升至87-90%，同时保留了超过90%的正确声明。AtomCite还具有迁移性：在冻结提示和零训练的情况下，它将两个开放7-8B模型在五个公共基准上的幻觉检测分数提升至高于同一模型作为直接评判员的水平。最后，审计显示自动标签中的噪声会偏置所测量的验证器准确率，并可能反转系统排名，因此仅依赖合成或自动标签的评估可能错误地衡量验证能力。
英文摘要
Large language models answering questions over multi-page documents are expected to cite the supporting pages, yet supplied citations are sometimes inaccurate, and current evaluations score citations at generation time or against text passages: no existing benchmark evaluates whether a system can verify and correct a page-level citation already attached to an answer. We propose AtomCite, an agentic framework that parses an answer into claims, checks each claim against the image of its cited page, and applies a deterministic repair policy. To evaluate it, we introduce DocCite, to our knowledge the first benchmark for systems that verify and correct page-level citations in document images. Built on MP-DocVQA and DUDE, it combines 928 validated injected instances with 2,468 candidate natural errors harvested from frontier- and efficiency-tier models, of which a two-annotator audit confirms 1,909 as genuine errors. Primary labels are assigned deterministically, not by LLM judges, with the human audit as a separate validation layer. Across three model families (Gemini, Claude, and GPT), AtomCite reaches around 93% binary verification accuracy on the injected benchmark, significantly outperforming every OCR-only condition, including a compute-matched control, and exceeding every prior text-based baseline given the same OCR text. Its repair policy lifts citation precision on the injected mix from a constructed 34% to 87-90% while retaining over 90% of correct claims. AtomCite also transfers: with frozen prompts and zero training, it raises the hallucination-detection scores of two open 7-8B models on five public benchmarks above the same models prompted as direct judges. Finally, the audit shows that noise in automatic labels biases measured verifier accuracy and can reverse system rankings, so evaluations relying only on synthetic or automatic labels risk mismeasuring verification capability.
