arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2505.00985cs.CL
收藏

立场:别再扩展LLM了!让我们聚焦于缩减规模

Position: Enough of Scaling LLMs! Lets Focus on Downscaling

Yash Goel, Ayan Sengupta, Tanmoy Chakraborty

AI总结:

本文挑战神经缩放定律的主导地位,提出缩减LLM规模的整体框架,以在保持性能的同时大幅降低资源需求,倡导可持续、高效且可及的开发范式。

摘要:

我们挑战对神经缩放定律的主导关注,并倡导在大型语言模型(LLM)开发中转向缩减规模的范式转变。虽然缩放定律通过增加模型和数据集规模为性能提升提供了关键见解,但我们强调这种方法的重要局限性,特别是在计算效率、环境影响和部署约束方面。为应对这些挑战,我们提出了一个用于缩减LLM的整体框架,旨在大幅降低资源需求的同时保持性能。本文概述了从传统缩放范式过渡的实用策略,倡导一种更可持续、高效且可及的LLM开发方法。

英文摘要:

We challenge the dominant focus on neural scaling laws and advocate for a paradigm shift toward downscaling in the development of large language models (LLMs). While scaling laws have provided critical insights into performance improvements through increasing model and dataset size, we emphasize the significant limitations of this approach, particularly in terms of computational inefficiency, environmental impact, and deployment constraints. To address these challenges, we propose a holistic framework for downscaling LLMs that seeks to maintain performance while drastically reducing resource demands. This paper outlines practical strategies for transitioning away from traditional scaling paradigms, advocating for a more sustainable, efficient, and accessible approach to LLM development.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑