arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2605.08583cs.CL
收藏

来源或未发生:一种用于引用伪造检测的多智能体框架

Source or It Didn't Happen: A Multi-Agent Framework for Citation Hallucination Detection

University of Massachusetts Amherst(马萨诸塞大学阿默斯特分校);The Ohio State University(俄亥俄州立大学)

Mingzhe Li, Zhiqiang Lin, Shiqing Ma

AI总结:

本文提出CiteTracer多智能体框架,通过12种分类体系检测引用伪造,利用PDF和BibTeX提取结构化引用,结合缓存查找、URL获取等方法验证,实现97.1%的准确率。

摘要:

大型语言模型在科学写作中被广泛应用,但它们可能生成看似合理却无法通过文献验证的引用。现有检测器通常将验证简化为二元判断,依赖脆弱的解析或不完整的检索,无法为审计者提供足够的领域信号。我们重新将引用伪造检测视为领域对齐的层面裁决,并引入涵盖真实、潜在和伪造引用的12种分类体系。基于此分类体系,我们构建了CiteTracer,一个级联多智能体检测器,能够从PDF和BibTeX中提取结构化引用,通过缓存查找、URL获取、学者连接器和网络搜索获取证据,应用确定性领域匹配,并将模糊情况转交给领域专家裁决。我们发布了一个包含2,450个合成引用的基准测试,这些引用基于真实的种子生成并受控的LLM变异,配对957个真实世界的伪造引用,来自ICLR 2026和匿名会议被拒提交。CiteTracer在合成基准测试上达到97.1%的准确率,领域级F1分数分别为真实、潜在和伪造引用的97.0、95.8和98.5,且在真实世界集上检测出97.1%的伪造引用,未出现回避情况。代码:https://github.com/aaFrostnova/CiteTracer。

英文摘要:

Large language models are increasingly used in scientific writing, yet they can fabricate citation-shaped references that appear plausible but fail bibliographic verification. Existing detectors often reduce verification to binary found/not-found decisions and rely on brittle parsing or incomplete retrieval, offering little field-level signal to auditors. We reframe citation hallucination detection as taxonomy-aligned field-level adjudication and introduce a 12-code taxonomy spanning Real, Potential, and Hallucinated citations. Based on this taxonomy, we build CiteTracer, a cascading multi-agent detector that extracts structured citations from PDF and BibTeX, retrieves evidence through cache lookup, URL fetch, scholar connectors, and web search, applies deterministic field matching, and routes ambiguous cases to class-specialist judgers. We release a benchmark of 2,450 synthetic citations built from real seeds with controlled LLM mutations, paired with 957 real-world fabricated citations drawn from desk-rejected submissions at ICLR 2026 and an anonymous conference. CiteTracer reaches 97.1% accuracy on the synthetic benchmark, with class-level F1 scores of 97.0, 95.8, and 98.5 for Real, Potential, and Hallucinated, respectively, and detects 97.1% of fabrications on the real-world set without abstaining. Code: https://github.com/aaFrostnova/CiteTracer.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑