arXivDaily arXiv每日学术速递 周一至周五更新
arXiv周末暂无论文更新,休息一下吧,周末愉快~~
arXiv 2409.17166cs.SEcs.AI
收藏

ScriptSmith:通过自动化 Bash 脚本生成、评估与改进增强 IT 运维的统一 LLM 框架

ScriptSmith: A Unified LLM Framework for Enhancing IT Operations via Automated Bash Script Generation, Assessment, and Refinement

IBM Research(IBM研究院);IBM Software(IBM软件部)

Oishik Chatterjee, Pooja Aggarwal, Suranjana Samanta, Ting Dai, Prateeti Mohapatra, Debanjana Kar, Ruchi Mahindru, Steve Barbieri, Eugen Postea, Brad Blancett, Arthur De Magalhaes

AI总结:

本文提出 ScriptSmith 统一 LLM 框架,通过自动化 Bash 脚本的生成、评估与改进来增强 IT 运维,实验表明该框架在脚本生成上实现了 7-10% 的总体提升,显著减少了人力并降低了对执行环境验证的依赖。

摘要:

在快速发展的站点可靠性工程(SRE)领域,对管理和解决站点及云应用程序问题的高效解决方案需求至关重要。本文提出了一种使用大型语言模型(LLM)进行脚本生成、评估和改进的动作自动化创新方法。通过利用 LLM 的能力,我们旨在显著减少编写和调试脚本所需的人力,从而提高 SRE 团队的生产力。我们的实验聚焦于 SRE 中常用的工具 Bash 脚本,并使用了包含 100 个任务的 CodeSift 数据集和包含 153 个任务的 InterCode 数据集。结果表明,LLM 能够自动评估和改进脚本,减少了在执行环境中进行脚本验证的需求。结果证明,该框架在脚本生成方面显示出 7-10% 的总体提升。

英文摘要:

In the rapidly evolving landscape of site reliability engineering (SRE), the demand for efficient and effective solutions to manage and resolve issues in site and cloud applications is paramount. This paper presents an innovative approach to action automation using large language models (LLMs) for script generation, assessment, and refinement. By leveraging the capabilities of LLMs, we aim to significantly reduce the human effort involved in writing and debugging scripts, thereby enhancing the productivity of SRE teams. Our experiments focus on Bash scripts, a commonly used tool in SRE, and involve the CodeSift dataset of 100 tasks and the InterCode dataset of 153 tasks. The results show that LLMs can automatically assess and refine scripts efficiently, reducing the need for script validation in an execution environment. Results demonstrate that the framework shows an overall improvement of 7-10% in script generation.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑