对比领域模型相似度指标与人类专家评分
Comparing Domain-Model Similarity Metrics Against Human Expert Ratings
浏览论文内容
中文总结 AI 辅助
本文实现5种领域模型相似度指标，对比其与39组领域模型的人类专家评分，发现无单一指标在所有标准占优，集成多指标或可替代人类专家评分。
中文摘要 AI 辅助
领域模型是模型驱动软件工程中的主要制品，它捕获利益相关者之间的共同理解，并作为下游软件开发的契约基础。这些语义模型的自动对比具有多种应用领域，例如需求工程、教育、领域模型自动生成、模型复用以及仓库挖掘。文献中提出了多种指标，但从业者尚无合理的方法在这些指标中进行选择。本文的贡献在于实现了5种此类指标，将它们应用于固定的39组领域模型对比任务，并将每种指标的输出与针对相同对比任务生成的人类专家评分进行对比。本文研究了两个研究问题：RQ1询问每种指标在39组对比任务中平均与人类专家评分的接近程度；RQ2询问每种指标在每组对比任务中与人类专家评分的距离一致性如何。研究结果表明，没有任何一种指标在所有标准上占据主导地位，相反，不同指标在个别标准上各自产生具有竞争力的结果——部分指标平均最接近人类专家评分，部分指标能最佳保留每组对比的排序——这表明结合多种指标的集成方法可能是人类专家评分的可行替代方案。本文的指标实现是该工作的一项制品，已按照FAIR4RS建议发布（DOI: https://doi.org/10.5281/zenodo.20942596）。
英文摘要
Domain models are a primary artefact in model-driven software engineering, where they capture the shared understanding between stakeholders and serve as the contractual basis for downstream software development. Automatic comparison of these semantic models has diverse application areas such as requirements engineering, education, automatic generation of domain models and model reuse and repository mining. The literature offers a variety of presented metrics, but for practitioners there is no defensible way to choose between them. The contribution of this paper is the implementation of five such metrics, their execution on a fixed set of 39 domain-model comparisons and the comparison of each metric's output against the human expert ratings produced for the same comparisons. Two research questions are addressed. RQ1 asks how close, on average, each metric is to the human expert rating across the 39 comparisons. RQ2 asks how consistent each metric's per-comparison distance from the human expert rating is. The findings reveal that no single metric achieves dominance across all criteria; rather, different metrics each yield competitive results on individual criteria - some closest on average, others best preserving the per-pair ordering - which suggests that an ensemble approach combining multiple metrics may serve as a viable substitute for human expert grading. The metric implementations are an artefact of this work and are published in accordance with the FAIR4RS recommendations (DOI: 10.5281/zenodo.20942596).
