TransformerRanker:一种高效寻找最适合下游分类任务的语言模型的工具
TransformerRanker: A Tool for Efficiently Finding the Best-Suited Language Models for Downstream Classification Tasks
AI总结:
本文提出开源轻量库TransformerRanker,通过LogME、H-Score、kNN可迁移性估计与层聚合,在无需微调的情况下为下游分类任务高效排序并选择最合适的PLM。
摘要:
NLP中的分类任务通常通过从模型中心选择一个预训练语言模型(PLM),并针对当前任务对其进行微调来解决。然而,鉴于目前可用的PLM数量非常多,一个实际挑战是确定其中哪一个在特定下游任务上表现最佳。在本文中,我们推出TransformerRanker,这是一个轻量级库,可高效地为分类任务对PLM进行排名,而无需计算成本高昂的微调。我们的库实现了当前的可迁移性估计方法(LogME、H-Score、kNN),并结合了层聚合选项;我们的实证研究表明,这些方法能够产生最先进的PLM排名(Garbas等人,2024)。我们将接口设计得轻量且易于使用,允许用户直接连接HuggingFace Transformers和Dataset库。用户只需选择一个下游分类任务和一组PLM,即可创建可能最适合其任务的PLM排名。我们将TransformerRanker作为可通过pip安装的开源库提供:https://github.com/flairNLP/transformer-ranker。
英文摘要:
Classification tasks in NLP are typically addressed by selecting a pre-trained language model (PLM) from a model hub, and fine-tuning it for the task at hand. However, given the very large number of PLMs that are currently available, a practical challenge is to determine which of them will perform best for a specific downstream task. With this paper, we introduce TransformerRanker, a lightweight library that efficiently ranks PLMs for classification tasks without the need for computationally costly fine-tuning. Our library implements current approaches for transferability estimation (LogME, H-Score, kNN), in combination with layer aggregation options, which we empirically showed to yield state-of-the-art rankings of PLMs (Garbas et al., 2024). We designed the interface to be lightweight and easy to use, allowing users to directly connect to the HuggingFace Transformers and Dataset libraries. Users need only select a downstream classification task and a list of PLMs to create a ranking of likely best-suited PLMs for their task. We make TransformerRanker available as a pip-installable open-source library https://github.com/flairNLP/transformer-ranker.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。