DOCFORGE-BENCH: A Comprehensive 0-shot Benchmark for Document Forgery Detection and Analysis
University of North Carolina at Chapel Hill（北卡罗来纳大学教堂山分校）；University of California, Irvine（加州大学尔湾分校）；Washington University in St. Louis（圣路易斯华盛顿大学）；Duke University（杜克大学）；University of California, Davis（加州大学戴维斯分校）；New York University（纽约大学）；Georgetown University（乔治城大学）
Zengqi Zhao, Weidi Xia, En Wei, Yan Zhang, Jane Mo, Tiannan Zhang, Yuanqin Dai, Zexi Chen, Yiran Tao, Simiao Ren
我们提出了DOCFORGE-BENCH，首个统一的零样本文档伪造检测基准，评估了14种方法在八个数据集上的表现，涵盖文本篡改、收据伪造和身份文档篡改。不同于以微调为导向的评估如ForensicHub [Du et al., 2025]，DOCFORGE-BENCH使用所有方法及其发表的预训练权重，并不进行领域适应——这是一个有意识的设计选择，反映了现实部署场景中从业者缺乏标注文档训练数据的情况。我们的核心发现是普遍存在校准失败问题，在单阈值协议下不可见：方法在像素AUC（≥0.76）上表现中等，但在像素F1上接近零。AUC-F1差距不是辨别失败，而是分数分布偏移：篡改区域在文档图像中只占0.27-4.17%的像素——比自然图像基准少一个数量级，使标准tau=0.5阈值灾难性地校准错误。Oracle-F1是固定阈值像素F1的2-10倍，证实校准而非表示是瓶颈。受控校准实验验证了这一点：在N=10个领域图像上调整单一阈值可恢复39-55%的Oracle-F1差距，证明阈值适应而非重新训练是实际部署的关键缺失步骤。总体而言，评估的方法在不同文档类型上均无法可靠地即开即用，强调文档伪造检测仍是一个未解问题。我们进一步指出，所有八个数据集都早于生成式AI编辑时代；涵盖扩散模型和大语言模型基于的文档伪造的基准代表了现代攻击面的关键开放缺口。
英文摘要：
We present DOCFORGE-BENCH, the first unified zero-shot benchmark for document forgery detection, evaluating 14 methods across eight datasets spanning text tampering, receipt forgery, and identity document manipulation. Unlike fine-tuning-oriented evaluations such as ForensicHub [Du et al., 2025], DOCFORGE-BENCH applies all methods with their published pretrained weights and no domain adaptation -- a deliberate design choice that reflects the realistic deployment scenario where practitioners lack labeled document training data. Our central finding is a pervasive calibration failure invisible under single-threshold protocols: methods achieve moderate Pixel-AUC (>=0.76) yet near-zero Pixel-F1. This AUC-F1 gap is not a discrimination failure but a score-distribution shift: tampered regions occupy only 0.27-4.17% of pixels in document images -- an order of magnitude less than in natural image benchmarks -- making the standard tau=0.5 threshold catastrophically miscalibrated. Oracle-F1 is 2-10x higher than fixed-threshold Pixel-F1, confirming that calibration, not representation, is the bottleneck. A controlled calibration experiment validates this: adapting a single threshold on N=10 domain images recovers 39-55% of the Oracle-F1 gap, demonstrating that threshold adaptation -- not retraining -- is the key missing step for practical deployment. Overall, no evaluated method works reliably out-of-the-box on diverse document types, underscoring that document forgery detection remains an unsolved problem. We further note that all eight datasets predate the era of generative AI editing; benchmarks covering diffusion- and LLM-based document forgeries represent a critical open gap on the modern attack surface.
ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis
SenseTime Research（商汤科技研究院）；Beijing University of Posts and Telecommunications（北京邮电大学）；Shanghai AI Laboratory（上海人工智能实验室）；Beihang University（北京航空航天大学）；University of Science and Technology of China（中国科学技术大学）；Nanyang Technological University（南洋理工大学）
TT-DF: A Large-Scale Diffusion-Based Dataset and Benchmark for Human Body Forgery Detection
MAIS \& NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China University of Science
So-Fake: Benchmarking and Explaining Social Media Image Forgery Detection
University of Liverpool, UK（利物浦大学）；Nanyang Technological University（南洋理工大学）；Zhejiang University（浙江大学）；National University of Singapore（新加坡国立大学）；The Hong Kong University of Science and Technology（香港科学与技术大学）；The Chinese University of Hong Kong, Shenzhen（香港中文大学（深圳））；The University of Sheffield（谢菲尔德大学）
Stacking Brick by Brick: Aligned Feature Isolation for Incremental Face Forgery Detection
School of Computer Science, Wuhan University（武汉大学计算机学院）；School of Electronic and Computer Engineering, Peking University Shenzhen Graduate School（北京大学深圳研究生院电子与计算机工程学院）；WeChat Vision, Tencent Inc.（腾讯公司微信视觉部）
MSN: Multi-directional Similarity Network for Hand-crafted and Deep-synthesized Copy-Move Forgery Detection
State Key Laboratory of Software Development Environment, Beihang University（软件开发环境国家重点实验室，北京航空航天大学）；Institute of Artificial Intelligence, Beihang University（人工智能研究院，北京航空航天大学）；State Key Laboratory of Information Security, Institute of Information Engineering, Chinese Academy of Sciences（信息安全部国家重点实验室，信息工程研究所，中国科学院）
MFFI: Multi-Dimensional Face Forgery Image Dataset for Real-World Scenarios
Anhui Province Key Laboratory of Digital Security（安徽省级数字安全重点实验室）；Hefei University of Technology（合肥工业大学）；Cardiff University（卡迪夫大学）；IHPC, Agency for Science, Technology and Research（科技研究局信息处理中心）；Beijing Academy of Artificial Intelligence（北京人工智能研究院）
The University of Sydney, Sydney, Australia（悉尼大学）；Xiamen University, Xiamen, China（厦门大学）；Hong Kong Polytechnic University, HongKong, China（香港理工大学）；National University of Singapore, Singapore, Singapore（新加坡国立大学）
Not Every Cue Is Evidence: Cue Supportability for Forensic Distillation in Face Forgery Detection
School of Software Technology, Zhejiang University（浙江大学软件学院）；Ant Group（蚂蚁集团）；College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；College of Computer Science and Technology, Zhejiang University of Technology（浙江工业大学计算机科学与技术学院）
Rethinking Image Forgery Detection via Soft Contrastive Learning and Unsupervised Clustering
School of Computer Science and Engineering, University of Electronic Science and Technology of China（电子科技大学计算机科学与工程学院）；Guangdong Key Laboratory of Intelligent Information Processing, College of Electronics and Information Engineering, Shenzhen University（广东智能信息处理重点实验室、电子与信息工程学院）
A War Beyond Deepfake: Benchmarking Facial Counterfeits and Countermeasures
Hanoi University of Science and Technology（河内理工大学）；Griffith University（格里菲斯大学）；Leibniz Universität Hannover（汉诺威莱布尼茨大学）；Deakin University（迪肯大学）；The University of Queensland（昆士兰大学）
LocateEdit-Bench: A Benchmark for Instruction-Based Editing Localization
Institute of Automation, Chinese Academy of Sciences, Beijing, China（中国科学院自动化研究所）；Beijing Academy of Artificial Intelligence, Beijing, China（北京人工智能研究院）；University of Chinese Academy of Sciences, Beijing, China（中国科学院大学）；School of EEECS, Queen's University Belfast, Northern Ireland, UK（女王大学 Belfast 电子工程与计算机科学学院）；Harbin Institute of Technology, Shenzhen, China（哈尔滨工业大学（深圳））；Peking University, Beijing, China（北京大学）
Weakly-supervised Audio Temporal Forgery Localization via Progressive Audio-language Co-learning Network
MoE Key Laboratory of Information Technology（信息科技联合实验室）；Sun Yat-sen University（中山大学）；State Key Laboratory of Mathematical Engineering and Advanced Computing（数学工程与先进计算国家重点实验室）；Alibaba Group（阿里巴巴集团）
Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Models
The University of Hong Kong（香港大学）；HKU Shanghai Intelligent Computing Research Center（港大上海智能计算研究中心）；Shanghai AI Laboratory（上海人工智能实验室）；Zhejiang Laboratory（浙江实验室）；Hangzhou Institute for Advanced Study（杭州高等研究院）；Zhejiang University（浙江大学）；Alibaba（阿里巴巴）；MEGVII Technology（旷视科技）
ManiVid: Unified and Explainable Forensic Analysis of Manipulated Videos
Shanghai Jiao Tong University（上海交通大学）；Beijing University of Posts and Telecommunications（北京邮电大学）；Fudan University（复旦大学）；The Chinese University of Hong Kong（香港中文大学）；Shanghai AI Laboratory（上海人工智能实验室）；Tsinghua University（清华大学）
ActivityForensics: A Comprehensive Benchmark for Localizing Manipulated Activity in Videos
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；School of Electrical and Electronic Engineering, Nanyang Technological University（南洋理工大学电气与电子工程学院）；School of Computing and Artificial Intelligence, Jiangxi University of Finance and Economics（江西财经大学计算机与人工智能学院）；Jiangxi Provincial Key Laboratory of Multimedia Intelligent Processing（江西省多媒体智能处理重点实验室）；Faculty of Engineering, Shenzhen MSU-BIT University（深圳北理莫斯科大学工程系）；VinUniversity
CoCoVideo: The High-Quality Commercial-Model-Based Contrastive Benchmark for AI-Generated Video Detection
School of Informatics, Xiamen University（厦门大学信息学院）；China Academy of Information and Communications Technology（中国信息通信技术研究院）；AI Transcend Pte. Ltd.（AI Transcend有限公司）
DFIL: Deepfake Incremental Learning by Exploiting Domain-invariant Forgery Clues
Zhejiang University（浙江大学）；ZJU-Hangzhou Global Scientific and Technological Innovation Center（浙江大学杭州国际科创中心）；Institute for Infocomm Research, A*STAR（新加坡科技研究局资讯通信研究院）；University College London（伦敦大学学院）
SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models
School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University（上海交通大学电子信息与电气工程学院）；Department of Artificial Intelligence, Xiamen University（厦门大学人工智能系）；School of Information Science and Technology, Shijiazhuang Tiedao University（石家庄铁道大学信息科学与技术学院）；Electrical Engineering Department, UCLA（加州大学洛杉矶分校电子工程系）；New Laboratory of Pattern Recognition(NLPR), Institute of Automation, Chinese Academy of Sciences（中国科学院自动化研究所模式识别新实验室）；College of Electronics and Information Engineering, Shenzhen University（深圳大学电子与信息工程学院）；School of Computing and Information Technology, Great Bay University（广东东莞Great Bay大学计算与信息科技学院）；School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University（中山大学深圳校区网络科学与技术学院）
PhotoHolmes: a Python library for forgery detection in digital images
Universidad de la República（共和国大学）；Facultad de Ingeniería（工程学院）；Centre Borelli（博雷利中心）；Ecole Normale Supérieure Paris-Saclay（巴黎-萨克雷高等师范学校）；Centro Pi, Instituto de Matemática Pura e Aplicada（纯数学与应用数学研究所皮中心）；Digital Sense（数字感知公司）
Chameleon: Benchmarking Detection and Backtracking on Commercial-Grade AI-Generated Videos
School of Computer Science and Technology, Guangdong University of Technology（广东工业大学计算机科学与技术学院）；School of Information Science and Technology, Guangdong University of Foreign Studies（广东外国语言文化大学信息科学与技术学院）
MM-VeriAgent: Learning to Use Extensive Tools to Verify Multimodal Misinformation with Reinforcement Learning
Beijing University of Posts and Telecommunications（北京邮电大学）；Minzu University of China（中央民族大学）；NLPR, Institute of automation, Chinese academy of science, Chinese Academy of Sciences（中国科学院自动化研究所模式识别国家重点实验室）
Low-rank Orthogonal Subspace Intervention for Generalizable Face Forgery Detection
Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology（数字取证工程研究中心，教育部，南京信息科技大学）
