arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2308.01413cs.CLcs.AI
收藏

LaFiCMIL:从相关多实例学习的视角重新思考大文件分类

LaFiCMIL: Rethinking Large File Classification from the Perspective of Correlated Multiple Instance Learning

University of Luxembourg(卢森堡大学);Luxembourg Institute of Science and Technology(卢森堡科学技术研究院);CentraleSupélec(中央高等电力学院)

Tiezhu Sun, Weiguo Pian, Nadia Daoudi, Kevin Allix, Tegawendé F. Bissyandé, Jacques Klein

AI总结:

LaFiCMIL从相关多实例学习视角解决大文件分类问题,在单GPU上高效扩展BERT至近2万令牌,并在七个基准数据集上取得最先进性能。

摘要:

基于Transformer的模型显著推进了自然语言处理,尤其是在文本分类任务中的表现。然而,这些模型在处理大文件时面临挑战,主要由于其输入限制,通常仅限于数百或数千个令牌。现有模型解决此问题的尝试通常包括从长输入中仅提取一小部分关键信息,同时由于其复杂架构而常常产生高计算成本。在本工作中,我们从相关多实例学习的视角解决大文件分类的挑战。我们引入了LaFiCMIL,一种专门为大文件分类设计的方法。LaFiCMIL针对在单个GPU上的高效运行进行了优化,使其成为二分类、多分类和多标签分类任务的多功能解决方案。我们使用七个多样且全面的基准数据集进行了广泛实验,以评估LaFiCMIL的有效性。通过集成BERT进行特征提取,LaFiCMIL展现了卓越的性能,在所有数据集上树立了新的基准。我们方法的一个显著成就是能够将BERT扩展到处理近20,000个令牌,同时在具有32GB内存的单个GPU上运行。这种效率,加上其最先进的性能,突显了LaFiCMIL作为大文件分类领域开创性方法的潜力。

英文摘要:

Transfomer-based models have significantly advanced natural language processing, in particular the performance in text classification tasks. Nevertheless, these models face challenges in processing large files, primarily due to their input constraints, which are generally restricted to hundreds or thousands of tokens. Attempts to address this issue in existing models usually consist in extracting only a fraction of the essential information from lengthy inputs, while often incurring high computational costs due to their complex architectures. In this work, we address the challenge of classifying large files from the perspective of correlated multiple instance learning. We introduce LaFiCMIL, a method specifically designed for large file classification. LaFiCMIL is optimized for efficient operation on a single GPU, making it a versatile solution for binary, multi-class, and multi-label classification tasks. We conducted extensive experiments using seven diverse and comprehensive benchmark datasets to assess LaFiCMIL's effectiveness. By integrating BERT for feature extraction, LaFiCMIL demonstrates exceptional performance, setting new benchmarks across all datasets. A notable achievement of our approach is its ability to scale BERT to handle nearly 20,000 tokens while operating on a single GPU with 32GB of memory. This efficiency, coupled with its state-of-the-art performance, highlights LaFiCMIL's potential as a groundbreaking approach in the field of large file classification.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑