Shell语言处理：用于机器学习的Unix命令解析
Shell Language Processing: Unix command parsing for Machine Learning
摘要：
在本文中，我们介绍了一个Shell语言预处理（SLP）库，它实现了针对解析Unix和Linux shell命令的标记化和编码。我们描述了需要一种新方法的理由，并给出了传统自然语言处理（NLP）管道失败的具体例子。此外，我们评估了我们的方法对广泛接受的信息和通信技术（ICT）标记化技术的安全分类任务，并实现了F1分数从0.392到0.874的显着改善。
英文摘要：
In this article, we present a Shell Language Preprocessing (SLP) library, which implements tokenization and encoding directed at parsing Unix and Linux shell commands. We describe the rationale behind the need for a new approach with specific examples of when conventional Natural Language Processing (NLP) pipelines fail. Furthermore, we evaluate our methodology on a security classification task against widely accepted information and communications technology (ICT) tokenization techniques and achieve significant improvement of an F1 score from 0.392 to 0.874.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
