面向视觉丰富文档的表格检测数据集再审视
Revisiting Table Detection Datasets for Visually Rich Documents
AI总结：
针对现有表格检测公开数据集噪声多、样本少、数据源有限导致评估不可靠的问题，本研究清理合并公开数据集构建Open-Tables，新增ICT领域的ICT-TD数据集，验证了其高可靠性与跨域适用性。
摘要：
随着电子文档数量激增，表格检测已成为视觉丰富文档理解的一项基础任务。然而，相关研究中广泛使用的主流公开数据集存在固有局限性，包括样本存在噪声且不一致、训练样本有限以及数据源单一等。这些缺陷导致此类数据集无法可靠地评估模型性能，也无法反映模型的真实能力。\n为此，本研究重新审视了一批带有高质量标注的公开数据集，识别并清理了其中的噪声，同时对齐这些数据集的标注定义，合并为一个更大规模的数据集，命名为Open-Tables。此外，为丰富数据源，我们利用信息与通信技术（ICT）商品的PDF文件构建了全新的ICT-TD数据集，该数据集属于不同领域，包含公开数据集中几乎不会出现的独特样本。为确保数据集的标注质量，我们在领域专家的指导下对数据集进行了人工标注。所提出的数据集具有较高的挑战性，可作为业务场景中实际案例的样本。\n我们利用多种主流的先进目标检测模型构建了强基线。实验结果表明，现有公开数据集尽管数据源不同，但领域间差异较小。我们提出的Open-Tables和ICT-TD凭借高质量且一致的标注，能够为模型提供更可靠的评估，同时也更适用于跨域场景。实验结果显示，在跨域设置下，使用清理后的Open-Tables数据集训练的基准模型，其加权平均F1值比使用含噪声版本Open-Tables训练的对应模型高出0.6%-2.6%，证明了所提出数据集的可靠性。这些数据集已公开可用。
英文摘要：
Table Detection has become a fundamental task for visually rich document understanding with the surging number of electronic documents. However, popular public datasets widely used in related studies have inherent limitations, including noisy and inconsistent samples, limited training samples, and limited data sources. These limitations make these datasets unreliable to evaluate the model performance and cannot reflect the actual capacity of models. Therefore, this study revisits some open datasets with high-quality annotations, identifies and cleans the noise, and aligns the annotation definitions of these datasets to merge a larger dataset, termed Open-Tables. Moreover, to enrich the data sources, we propose a new ICT-TD dataset using the PDF files of Information and Communication Technologies (ICT) commodities, a different domain containing unique samples that hardly appear in open datasets. To ensure the label quality of the dataset, we annotated the dataset manually following the guidance of a domain expert. The proposed dataset is challenging and can be a sample of actual cases in the business context. We built strong baselines using various state-of-the-art object detection models. Our experimental results show that the domain differences among existing open datasets are minor despite having different data sources. Our proposed Open-Tables and ICT-TD can provide a more reliable evaluation for models because of their high quality and consistent annotations. Besides, they are more suitable for cross-domain settings. Our experimental results show that in the cross-domain setting, benchmark models trained with cleaned Open-Tables dataset can achieve 0.6\%-2.6\% higher weighted average F1 than the corresponding ones trained with the noisy version of Open-Tables, demonstrating the reliability of the proposed datasets. The datasets are public available.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
