arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.16559cs.SEcs.AI
收藏

揭示LLM开源项目中的问题、原因及解决方案

Demystifying Issues, Causes and Solutions in LLM Open-Source Projects

School of Computer Science, Wuhan University(武汉大学计算机学院);School of Computer Science & Hubei Provincial Key Laboratory of Artificial Intelligence and Smart Learning, Central China Normal University(华中师范大学计算机学院);School of Computing Technologies, RMIT University(RMIT大学计算技术学院)

Yangxiao Cai, Peng Liang, Yifei Wang, Zengyang Li, Mojtaba Shahin

AI总结:

针对LLM开源项目缺乏挑战与解决方案研究的问题,本文通过分析15个项目的994个issue,发现模型问题是最常见问题,优化模型是主要解决方案,并为从业者和研究人员提供了启示。

摘要:

随着大型语言模型(LLMs)的进步,越来越多的开源软件项目将LLMs作为其核心功能组件。尽管关于LLMs的研究与实践引起了广泛关注,但目前尚无专门研究探讨LLM开源项目从业者面临的挑战、这些挑战的原因以及潜在的解决方案。为了填补这一研究空白,我们进行了一项实证研究,以了解从业者在开发和使LLM开源软件时遇到的问题、这些问题的可能原因以及潜在的解决方案。我们从15个LLM开源项目中收集了所有已关闭的issue,并对符合我们要求的issue进行了标记。随后,我们从标记的issue中随机抽取了994个issue作为数据提取和分析的样本,以了解普遍存在的问题、其根本原因以及潜在的解决方案。我们的研究结果表明:(1)模型问题是从业者面临的最常见问题;(2)模型问题、配置与连接问题以及特征与方法问题被确定为最频繁的问题原因;(3)优化模型是解决这些问题的主要方案。基于研究结果,我们为LLM开源项目的从业者和研究人员提供了实践启示。

英文摘要:

With the advancements of Large Language Models (LLMs), an increasing number of open-source software projects are using LLMs as their core functional component. Although research and practice on LLMs are capturing considerable interest, no dedicated studies explored the challenges faced by practitioners of LLM open-source projects, the causes of these challenges, and potential solutions. To fill this research gap, we conducted an empirical study to understand the issues that practitioners encounter when developing and using LLM open-source software, the possible causes of these issues, and potential solutions. We collected all closed issues from 15 LLM open-source projects and labelled issues that met our requirements. We then randomly selected 994 issues from the labelled issues as the sample for data extraction and analysis to understand the prevalent issues, their underlying causes, and potential solutions. Our study results show that (1) Model Issue is the most common issue faced by practitioners, (2) Model Problem, Configuration and Connection Problem, and Feature and Method Problem are identified as the most frequent causes of the issues, and (3) Optimize Model is the predominant solution to the issues. Based on the study results, we provide implications for practitioners and researchers of LLM open-source projects.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑