arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2602.07238cs.AIcs.LGecon.GNq-fin.EC
收藏

大型语言模型开发中是否存在'秘密配方'?

Is there "Secret Sauce'' in Large Language Model Development?

Massachusetts Institute of Technology(麻省理工学院)

Matthias Mertens, Natalia Fischl-Lanzoni, Neil Thompson

AI总结:

研究发现大型语言模型的前沿进展主要由计算量驱动,而非专有技术,但非前沿领域中专有技术可显著降低计算需求。

摘要:

领先的大语言模型(LLM)开发者是否拥有专有技术

英文摘要:

Do leading LLM developers possess a proprietary ``secret sauce'', or is LLM performance driven by scaling up compute? Using training and benchmark data for 809 models released between 2022 and 2025, we estimate scaling-law regressions with release-date and developer fixed effects. We find clear evidence of developer-specific efficiency advantages, but their importance depends on where models lie in the performance distribution. At the frontier, 80-90% of performance differences are explained by higher training compute, implying that scale--not proprietary technology--drives frontier advances. Away from the frontier, however, proprietary techniques and shared algorithmic progress substantially reduce the compute required to reach fixed capability thresholds. Some companies can systematically produce smaller models more efficiently. Strikingly, we also find substantial variation of model efficiency within companies; a firm can train two models with more than 40x compute efficiency difference. We also discuss the implications for AI leadership and capability diffusion.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑