IndicDLP: 一个多语言多领域文档布局解析基础数据集
IndicDLP: A Foundational Dataset for Multi-Lingual and Multi-Domain Document Layout Parsing
AI总结：
IndicDLP是一个多语言多领域文档布局解析数据集，旨在解决现有数据集在规模、多样性和标注细粒度方面的不足，提升文档理解的效率和包容性。
摘要：
文档布局分析对于信息检索、提取、OCR和数字化等下游任务至关重要。然而，现有的大规模数据集如PubLayNet和DocBank缺乏细粒度区域标签和多语言多样性，使其无法代表复杂的文档布局。相比之下，人工标注的数据集如M6Doc和D4LA提供了更丰富的标签和更大的领域多样性，但数据量太小，无法训练出鲁棒的模型，且缺乏充分的多语言覆盖。这种差距在印地语文档中尤为明显，因为它们包含多种脚本，但在当前数据集中仍然代表性不足，进一步限制了该领域的发展。为了解决这些不足，我们引入了IndicDLP，一个覆盖11种代表性印地语语言以及英语和12种常见文档领域的大型基础文档布局数据集。此外，我们还整理了UED-mini数据集，该数据集源自DocLayNet和M6Doc，以增强预训练并为印地语布局模型提供坚实的基础。我们的实验表明，对现有英文模型进行微调以使用IndicDLP显著提高了性能，验证了其有效性。此外，训练于IndicDLP的模型在印地语布局之外也能很好地推广，使其成为文档数字化的宝贵资源。这项工作填补了规模、多样性和标注细粒度之间的空白，推动了包容性和高效的文档理解。
英文摘要：
Document layout analysis is essential for downstream tasks such as information retrieval, extraction, OCR, and digitization. However, existing large-scale datasets like PubLayNet and DocBank lack fine-grained region labels and multilingual diversity, making them insufficient for representing complex document layouts. In contrast, human-annotated datasets such as M6Doc and D4LA offer richer labels and greater domain diversity, but are too small to train robust models and lack adequate multilingual coverage. This gap is especially pronounced for Indic documents, which encompass diverse scripts yet remain underrepresented in current datasets, further limiting progress in this space. To address these shortcomings, we introduce IndicDLP, a large-scale foundational document layout dataset spanning 11 representative Indic languages alongside English and 12 common document domains. Additionally, we curate UED-mini, a dataset derived from DocLayNet and M6Doc, to enhance pretraining and provide a solid foundation for Indic layout models. Our experiments demonstrate that fine-tuning existing English models on IndicDLP significantly boosts performance, validating its effectiveness. Moreover, models trained on IndicDLP generalize well beyond Indic layouts, making it a valuable resource for document digitization. This work bridges gaps in scale, diversity, and annotation granularity, driving inclusive and efficient document understanding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
