arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2410.01166cs.CL
收藏

使用文件名进行文档分类

Document Classification using File Names

Vanderbilt University(范德堡大学)

Zhijian Li, Stefan Larson, Kevin Leach

AI总结:

针对传统深度学习文档分类推理慢、资源消耗大的问题,提出基于轻量级监督学习和 TF-IDF 的文件名分类方法,在保持高准确率的同时大幅提升处理速度。

摘要:

快速文档分类在数字取证和大规模媒体分类等时间敏感型应用中至关重要。依赖重量级深度学习模型的传统方法由于在庞大输入数据集上的高推理时间以及分析整篇文档所需的计算资源而表现不佳。在本文中,我们提出了一种使用轻量级监督学习模型,结合基于 TF-IDF 特征提取的词元化方法,仅根据文件名即可准确高效地对文档进行分类的方法,该方法大幅减少了推理时间。我们的结果表明,文件名分类器在两个数据集上测试时,能够处理超过 90% 的范围内文档,准确率分别达到 99.63% 和 96.57%,同时比 DiT 等更复杂的模型快 442 倍。我们的方法为在关键场景中高效处理庞大文档数据集提供了关键解决方案,实现了更快速、更可靠的文档分类。

英文摘要:

Rapid document classification is critical in several time-sensitive applications like digital forensics and large-scale media classification. Traditional approaches that rely on heavy-duty deep learning models fall short due to high inference times over vast input datasets and computational resources associated with analyzing whole documents. In this paper, we present a method using lightweight supervised learning models, combined with a TF-IDF feature extraction-based tokenization method, to accurately and efficiently classify documents based solely on file names, that substantially reduces inference time. Our results indicate that file name classifiers can process more than 90% of in-scope documents with 99.63% and 96.57% accuracy when tested on two datasets, while being 442x faster than more complex models such as DiT. Our method offers a crucial solution to efficiently process vast document datasets in critical scenarios, enabling fast and more reliable document classification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑