arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2601.22157cs.LGcs.CL
收藏

在模型仓库中发现隐藏的瑰宝

Discovering Hidden Gems in Model Repositories

Department of Computer Science(计算机科学系);School of Computer Science and Engineering(计算机科学与工程学院);The Hebrew University of Jerusalem(特拉维夫大学)

Jonathan Kahana, Eliahu Horwitz, Yedid Hoshen

AI总结:

通过多臂老虎机问题加速模型发现,发现大量性能优异但不受欢迎的模型。

摘要:

公共仓库托管了数以百万计的微调模型,但社区使用仍然 disproportionately 集中在少数几个基础检查点上。我们调查这种集中是否反映了高效的市场选择,还是更优模型被系统性地忽视。通过对超过2000个模型的广泛评估,我们展示了“隐藏瑰宝”的普遍性,即不受欢迎的微调模型在性能上显著优于其受欢迎的对应物。值得注意的是,在Llama-3.1-8B系列中,我们发现了很少被下载的检查点,其数学性能从83.2%提升到96.0%,而推理成本并未增加。然而,通过对每个上传模型进行彻底评估来发现这些模型在计算上是不可行的。因此,我们将模型发现建模为多臂老虎机问题,并通过使用共享查询集和激进的淘汰计划来加速顺序削减搜索算法。我们的方法在每个候选者上仅需50次查询即可检索到顶级模型,将发现速度提高了超过50倍。

英文摘要:

Public repositories host millions of fine-tuned models, yet community usage remains disproportionately concentrated on a small number of foundation checkpoints. We investigate whether this concentration reflects efficient market selection or if superior models are systematically overlooked. Through an extensive evaluation of over 2,000 models, we show the prevalence of "hidden gems", unpopular fine-tunes that significantly outperform their popular counterparts. Notably, within the Llama-3.1-8B family, we find rarely downloaded checkpoints that improve math performance from 83.2% to 96.0% without increasing inference costs. However, discovering these models through exhaustive evaluation of every uploaded model is computationally infeasible. We therefore formulate model discovery as a Multi-Armed Bandit problem and accelerate the Sequential Halving search algorithm by using shared query sets and aggressive elimination schedules. Our method retrieves top models with as few as 50 queries per candidate, accelerating discovery by over 50x.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑