XMainframe：用于大型机现代化的大型语言模型
XMainframe: A Large Language Model for Mainframe Modernization
AI总结：
本文提出XMainframe，一个专为大型机遗留系统和COBOL代码库设计的大型语言模型，并构建了高质量训练数据集与综合基准MainframeBench，在多项任务上显著超越现有模型，推动遗留系统现代化。
摘要：
大型机操作系统尽管起源于20世纪40年代，但仍然继续支持金融和政府等关键领域。然而，这些系统通常被视为过时，需要广泛的维护和现代化。应对这一挑战需要能够理解并与遗留代码库交互的创新工具。为此，我们引入了XMainframe，这是一种最先进的大型语言模型（LLM），专门设计用于掌握大型机遗留系统和COBOL代码库的知识。我们的解决方案包括创建一个广泛的数据收集管道，以生成高质量的训练数据集，从而提升XMainframe在这一专业领域的性能。此外，我们提出了MainframeBench，这是一个用于评估大型机知识的综合基准，包括多项选择题、问答和COBOL代码摘要。我们的实证评估表明，XMainframe在这些任务中始终优于现有的最先进LLM。具体而言，XMainframe在多项选择题上的准确率比DeepSeek-Coder高30%，在问答上的BLEU分数是Mixtral-Instruct 8x7B的两倍，在COBOL摘要上的得分是GPT-3.5的六倍。我们的工作突显了XMainframe在推动遗留系统管理和现代化方面取得重大进展的潜力，从而提高生产力并为软件开发人员节省时间。
英文摘要：
Mainframe operating systems, despite their inception in the 1940s, continue to support critical sectors like finance and government. However, these systems are often viewed as outdated, requiring extensive maintenance and modernization. Addressing this challenge necessitates innovative tools that can understand and interact with legacy codebases. To this end, we introduce XMainframe, a state-of-the-art large language model (LLM) specifically designed with knowledge of mainframe legacy systems and COBOL codebases. Our solution involves the creation of an extensive data collection pipeline to produce high-quality training datasets, enhancing XMainframe's performance in this specialized domain. Additionally, we present MainframeBench, a comprehensive benchmark for assessing mainframe knowledge, including multiple-choice questions, question answering, and COBOL code summarization. Our empirical evaluations demonstrate that XMainframe consistently outperforms existing state-of-the-art LLMs across these tasks. Specifically, XMainframe achieves 30% higher accuracy than DeepSeek-Coder on multiple-choice questions, doubles the BLEU score of Mixtral-Instruct 8x7B on question answering, and scores six times higher than GPT-3.5 on COBOL summarization. Our work highlights the potential of XMainframe to drive significant advancements in managing and modernizing legacy systems, thereby enhancing productivity and saving time for software developers.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
