arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2312.06121cs.SEcs.AI
收藏

LLM 能否配置软件工具

Can LLMs Configure Software Tools

Applied Artificial Intelligence Institute(应用人工智能研究院);Deakin University(迪肯大学)

Jai Kannan

AI总结:

本文研究利用 Chat-GPT 等 LLM 为机器学习组件确定超参数初始条件并缩小搜索空间,实验表明其可提升配置与优化效率,但仍需深入验证。

摘要:

在软件工程中,对软件工具进行细致配置,对于确保复杂系统内的最佳性能至关重要。然而,现代应用所呈现的高维搜索空间加剧了选择最优配置本身固有的复杂性。传统的试错法或直觉驱动方法既低效又容易出错,阻碍了可扩展性和可复现性。在本研究中,我们探索利用 Large-Language Models(LLM,大语言模型)来简化软件配置流程。我们发现,由于搜索空间庞大且性能至关重要,智能应用中机器学习组件的超参数配置任务尤其具有挑战性。包括贝叶斯优化在内的现有方法在初始设置、计算成本和收敛效率方面存在局限。我们的工作提出了一种新颖方法,利用 Chat-GPT 等 LLM 来确定起始条件并缩小搜索空间,从而提高配置效率。我们开展了一系列实验,考察 LLM 生成响应的变异性,并揭示了一些有趣发现,例如潜在的响应缓存以及基于领域特定关键词的一致行为。此外,超参数优化实验结果表明,LLM 在加速初始化过程和优化配置方面具有潜力。尽管我们的初步见解令人鼓舞,但它们也表明需要在该领域开展更深入的研究和实验。

英文摘要:

In software engineering, the meticulous configuration of software tools is crucial in ensuring optimal performance within intricate systems. However, the complexity inherent in selecting optimal configurations is exacerbated by the high-dimensional search spaces presented in modern applications. Conventional trial-and-error or intuition-driven methods are both inefficient and error-prone, impeding scalability and reproducibility. In this study, we embark on an exploration of leveraging Large-Language Models (LLMs) to streamline the software configuration process. We identify that the task of hyperparameter configuration for machine learning components within intelligent applications is particularly challenging due to the extensive search space and performance-critical nature. Existing methods, including Bayesian optimization, have limitations regarding initial setup, computational cost, and convergence efficiency. Our work presents a novel approach that employs LLMs, such as Chat-GPT, to identify starting conditions and narrow down the search space, improving configuration efficiency. We conducted a series of experiments to investigate the variability of LLM-generated responses, uncovering intriguing findings such as potential response caching and consistent behavior based on domain-specific keywords. Furthermore, our results from hyperparameter optimization experiments reveal the potential of LLMs in expediting initialization processes and optimizing configurations. While our initial insights are promising, they also indicate the need for further in-depth investigations and experiments in this domain.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑