arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.09334cs.CL
收藏

ProxyLM:通过代理模型预测语言模型在多语言任务上的性能

ProxyLM: Predicting Language Model Performance on Multilingual Tasks via Proxy Models

University of Toronto(多伦多大学);Capital One(第一资本金融公司);Institut Teknologi Bandung(万隆理工学院);Ontario Tech University(安大略理工大学)

David Anugraha, Genta Indra Winata, Chenyue Li, Patrick Amadeus Irawan, En-Shiun Annie Lee

AI总结:

ProxyLM提出一种可扩展的任务和语言无关框架,利用代理模型预测语言模型性能,实现高达37.08倍加速,并在RMSE上超越现有最优方法至少1.78倍。

摘要:

性能预测是一种估计语言模型(LMs)在各种自然语言处理(NLP)任务上性能的方法,可减轻与模型容量和微调数据相关的计算成本。我们的论文提出了ProxyLM,一个可扩展的、任务和语言无关的框架,旨在使用代理模型预测语言模型的性能。这些代理模型作为替代品,近似于所关注的语言模型的性能。通过利用这些代理模型,ProxyLM显著降低了任务评估中的计算开销,与传统方法相比,即使使用我们最小的代理模型,也能实现高达37.08倍的加速。我们在多个多语言NLP任务和各种鲁棒性测试上的结果表明,ProxyLM不仅能够很好地适应预训练语言模型中未见过的语言,还能在不同数据集上有效泛化,在均方根误差(RMSE)方面至少比现有最优方法提升1.78倍。

英文摘要:

Performance prediction is a method to estimate the performance of Language Models (LMs) on various Natural Language Processing (NLP) tasks, mitigating computational costs associated with model capacity and data for fine-tuning. Our paper presents ProxyLM, a scalable task- and language-agnostic framework designed to predict the performance of LMs using proxy models. These proxy models act as surrogates, approximating the performance of the LM of interest. By leveraging these proxy models, ProxyLM significantly reduces computational overhead in task evaluations, achieving up to a 37.08x speedup over traditional methods, even with our smallest proxy models. Our results across multiple multilingual NLP tasks and various robustness tests demonstrate that ProxyLM not only adapts well to previously unseen languages in pre-trained LMs, but also generalizes effectively across different datasets, outperforming the state-of-the-art by at least 1.78x in terms of root-mean-square error (RMSE).

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑