VeriTaS：首个动态多模态自动事实核查基准
VeriTaS: The First Dynamic Benchmark for Multimodal Automated Fact-Checking
AI总结：
为应对在线虚假信息的扩大规模，本文提出VeriTaS动态基准，通过自动化流程持续更新，涵盖104个专业机构的25000个真实声明，支持多模态事实核查评估。
摘要：
随着在线虚假信息规模的扩大，自动事实核查（AFC）系统评估亟需动态基准。现有基准在任务范围、模态、领域、语言多样性、真实性及虚假信息类型覆盖方面存在局限，且静态特性导致数据泄漏问题。本文引入VeriTaS，首个动态多模态AFC基准，通过自动化流程持续更新，涵盖104个专业机构的25000个声明，覆盖文本和音视频内容。声明每季度通过七阶段自动化流程更新，标准化声明表述、检索原始媒体并映射异构专家判断至标准化评分方案。通过人工评估证明自动化标注与人类判断一致。未来将持续更新，建立抗数据泄漏的基准，支持快速演进的基础模型时代的AFC评估。代码和数据公开于https://veritas.mai.informatik.tu-darmstadt.de。
英文摘要：
The growing scale of online misinformation urgently demands Automated Fact-Checking (AFC). Existing benchmarks for evaluating AFC systems, however, are largely limited in terms of task scope, modalities, domain, language diversity, realism, or coverage of misinformation types. Critically, they are static, thus subject to data leakage as their claims enter the pretraining corpora of LLMs. As a result, benchmark performance no longer reliably reflects the actual ability to verify claims. We introduce Verified Theses and Statements (VeriTaS), the first dynamic benchmark for multimodal AFC, designed to remain robust under ongoing large-scale pretraining of foundation models. VeriTaS currently comprises 25,000 real-world claims from 104 professional fact-checking organizations across 54 languages, covering textual and audiovisual content. Claims are added quarterly via a fully automated seven-stage pipeline that normalizes claim formulation, retrieves original media, and maps heterogeneous expert verdicts to a novel, standardized, and disentangled scoring scheme with textual justifications. Through human evaluation, we demonstrate that the automated annotations closely match human judgments. We commit to updating VeriTaS in the future, establishing a leakage-resistant benchmark, supporting meaningful AFC evaluation in the era of rapidly evolving foundation models. The code and data are publicly available under https://veritas.mai.informatik.tu-darmstadt.de .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
