Harbsafe-162.一种用于术语数据语义表示内在评估的特定领域数据集
Harbsafe-162. A Domain-Specific Data Set for the Intrinsic Evaluation of Semantic Representations for Terminological Data
摘要：
本文介绍了Harbsafe-162，一个用于评估分布式语义模型的特定于领域的数据集。它起源于布伦瑞克工业大学和DIN和VDE的德国电气、电子和信息技术委员会的合作，即Harbsafe项目。该项目的一个目标是将分布式语义模型应用于术语条目，即由至少一个或多个术语、术语短语和定义组成的复杂词汇数据。需要这种应用程序来解决一个更复杂的问题：标准和标准机构术语的协调（即解决双重和不一致）。由于缺乏术语条目的评价数据集，创建Harbsafe-162是统一援助的必要步骤。Harbsafe-162涵盖了功能安全、IT安全和可靠性领域的九个电工标准的数据。一个内在的评价方法，在一个相似性评级任务的形式已被应用，其中两个语言学家和三个领域的专家从标准化参与。该数据集用于评估所建立的句子嵌入模型的具体实现。这一实施证明对特定领域的数据是令人满意的，因此，该项目可以提出进一步的协调援助实施。考虑到最近的批评内在的评价方法，本文的结论与Harbsafe-162的评价，并加入了更广泛的讨论相似性评级任务的性质。Harbsafe-162已向社区提供。
英文摘要：
The article presents Harbsafe-162, a domain-specific data set for evaluating distributional semantic models. It originates from a cooperation by Technische Universität Braunschweig and the German Commission for Electrical, Electronic & Information Technologies of DIN and VDE, the Harbsafe project. One objective of the project is to apply distributional semantic models to terminological entries, that is, complex lexical data comprising of at least one or several terms, term phrases and a definition. This application is needed to solve a more complex problem: the harmonization of terminologies of standards and standards bodies (i.e. resolution of doublettes and inconsistencies). Due to a lack of evaluation data sets for terminological entries, the creation of Harbsafe-162 was a necessary step towards harmonization assistance. Harbsafe-162 covers data from nine electrotechnical standards in the domain of functional safety, IT security, and dependability. An intrinsic evaluation method in the form of a similarity rating task has been applied in which two linguists and three domain experts from standardization participated. The data set is used to evaluate a specific implementation of an established sentence embedding model. This implementation proves to be satisfactory for the domain-specific data so that further implementations for harmonization assistance may be brought forward by the project. Considering recent criticism on intrinsic evaluation methods, the article concludes with an evaluation of Harbsafe-162 and joins a more general discussion about the nature of similarity rating tasks. Harbsafe-162 has been made available for the community.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
