Secu-Table：一个用于评估语义表格解释系统的综合安全表格数据集
Secu-Table: a Comprehensive security table dataset for evaluating semantic table interpretation systems
AI总结：
本文提出Secu-Table数据集，包含1500多个安全领域表格和15k多个实体，用于在SemTab挑战中评估基于LLM的语义表格解释系统。
摘要：
评估语义表格解释（STI）系统（尤其是基于大语言模型（LLM）的系统），在安全领域等特定领域情境下，高度依赖于数据集。然而，在安全领域，面向最先进技术的表格数据集并未公开可用。在本文中，我们介绍了Secu-Table数据集，该数据集由1500多个表格和超过15k个实体组成，使用从通用漏洞披露（CVE）和通用弱点枚举（CWE）数据源中提取的安全数据构建，并使用Wikidata和SEmantic Processing of Security Event Streams网络安全知识图谱（SEPSES CSKG）进行标注。随数据集一起，所有代码均已公开发布。该数据集在SemTab挑战赛的表格到知识图谱匹配任务背景下提供给研究社区。该挑战赛旨在评估多个基于开源LLM的STI的性能。作为基线的初步评估使用Falcon3-7b-instruct和Mistral-7B-Instruct这两个开源LLM以及GPT-4o mini这一个闭源LLM进行。
英文摘要：
Evaluating semantic tables interpretation (STI) systems, (particularly, those based on Large Language Models- LLMs) especially in domain-specific contexts such as the security domain, depends heavily on the dataset. However, in the security domain, tabular datasets for state-of-the-art are not publicly available. In this paper, we introduce Secu-Table dataset, composed of more than 1500 tables with more than 15k entities constructed using security data extracted from Common Vulnerabilities and Exposures (CVE) and Common Weakness Enumeration (CWE) data sources and annotated using Wikidata and the SEmantic Processing of Security Event Streams CyberSecurity Knowledge Graph (SEPSES CSKG). Along with the dataset, all the code is publicly released. This dataset is made available to the research community in the context of the SemTab challenge on Tabular to Knowledge Graph Matching. This challenge aims to evaluate the performance of several STI based on open source LLMs. Preliminary evaluation, serving as baseline, was conducted using Falcon3-7b-instruct and Mistral-7B-Instruct, two open source LLMs and GPT-4o mini one closed source LLM.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
