arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2305.02783cs.SEcs.AIcs.CLcs.PL
收藏

通过大型语言模型自动生成信息技术任务的 YAML 代码

Automated Code generation for Information Technology Tasks in YAML through Large Language Models

IBM Research(IBM研究院);Red Hat(红帽公司)

Saurabh Pujar, Luca Buratti, Xiaojie Guo, Nicolas Dupuis, Burn Lewis, Sahil Suneja, Atin Sood, Ganesh Nalawade, Matthew Jones, Alessandro Morari, Ruchir Puri

AI总结:

本文提出基于 transformer 的自然语言到 Ansible-YAML 代码生成工具 Ansible Wisdom,通过新数据集训练并开发针对 YAML 的新指标,实验表明其性能优于 Codex-Davinci-002 等现有模型。

摘要:

由于使用大型语言模型,代码生成能力的近期提升主要惠及通用编程语言。领域特定语言,例如用于 IT 自动化的语言,尽管涉及许多活跃开发者并且是现代云平台的重要组成部分,却受到的关注要少得多。这项工作专注于生成 Ansible-YAML,这是一种广泛用于 IT 自动化的标记语言。我们提出了 Ansible Wisdom,这是一个从自然语言到 Ansible-YAML 的代码生成工具,旨在提高 IT 自动化生产力。Ansible Wisdom 是一个基于 transformer 的模型,通过使用包含 Ansible-YAML 的新数据集进行训练来扩展。我们还开发了两个针对 YAML 和 Ansible 的新性能指标,以捕捉该领域的特定特征。结果表明,Ansible Wisdom 能够准确地从自然语言提示生成 Ansible 脚本,其性能可与现有的最先进代码生成模型相当或更好。在少样本设置中,我们评估了使用 Ansible、YAML 数据进行训练的影响,并与包括 Codex-Davinci-002 在内的不同基线进行了比较。我们还表明,在微调之后,我们特定于 Ansible 的模型(BLEU:66.67)可以优于大得多的 Codex-Davinci-002(BLEU:50.4)模型,后者是在少样本设置下评估的。

英文摘要:

The recent improvement in code generation capabilities due to the use of large language models has mainly benefited general purpose programming languages. Domain specific languages, such as the ones used for IT Automation, have received far less attention, despite involving many active developers and being an essential component of modern cloud platforms. This work focuses on the generation of Ansible-YAML, a widely used markup language for IT Automation. We present Ansible Wisdom, a natural-language to Ansible-YAML code generation tool, aimed at improving IT automation productivity. Ansible Wisdom is a transformer-based model, extended by training with a new dataset containing Ansible-YAML. We also develop two novel performance metrics for YAML and Ansible to capture the specific characteristics of this domain. Results show that Ansible Wisdom can accurately generate Ansible script from natural language prompts with performance comparable or better than existing state of the art code generation models. In few-shot settings we asses the impact of training with Ansible, YAML data and compare with different baselines including Codex-Davinci-002. We also show that after finetuning, our Ansible specific model (BLEU: 66.67) can outperform a much larger Codex-Davinci-002 (BLEU: 50.4) model, which was evaluated in few shot settings.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑