跨语言评估维基数据源的质量:一种混合方法
Assessing the quality of sources in Wikidata across languages: a hybrid approach
摘要:
维基数据是网络上最重要的结构化数据来源之一,由全球志愿者社区建立。作为第二来源,其内容必须有可靠的参考文献支持;这一点尤其重要,因为维基数据明确鼓励编辑添加没有广泛共识的声明,只要它们得到参考文献的证实。尽管如此,尽管内容和参考文献之间存在这种重要联系,维基数据系统评估和确保其参考文献质量的能力仍然有限。为此,我们进行了一项混合方法研究,以确定维基数据参考文献的相关性,易用性和权威性,在规模和不同的语言中,使用在线众包,描述性统计和机器学习。在我们以前的工作的基础上,我们运行了一系列微任务实验来评估一个大型的参考文献语料库,这些参考文献是从具有多种语言标签的维基数据三元组中采样的。我们使用一个整合的、策划的众包评估版本来训练几个机器学习模型,以将分析扩展到整个维基数据。这些发现帮助我们确定维基数据中引用的质量,并确定在定义和捕获网络上用户生成的多语言结构化数据的质量方面的常见挑战。我们还讨论了正在进行的编辑实践,这可以鼓励以更直接的方式使用更高质量的参考资料。研究中使用的所有数据和代码都可以在GitHub上获得反馈,并由研究社区进一步改进和部署。
英文摘要:
Wikidata is one of the most important sources of structured data on the web, built by a worldwide community of volunteers. As a secondary source, its contents must be backed by credible references; this is particularly important as Wikidata explicitly encourages editors to add claims for which there is no broad consensus, as long as they are corroborated by references. Nevertheless, despite this essential link between content and references, Wikidata's ability to systematically assess and assure the quality of its references remains limited. To this end, we carry out a mixed-methods study to determine the relevance, ease of access, and authoritativeness of Wikidata references, at scale and in different languages, using online crowdsourcing, descriptive statistics, and machine learning. Building on previous work of ours, we run a series of microtasks experiments to evaluate a large corpus of references, sampled from Wikidata triples with labels in several languages. We use a consolidated, curated version of the crowdsourced assessments to train several machine learning models to scale up the analysis to the whole of Wikidata. The findings help us ascertain the quality of references in Wikidata, and identify common challenges in defining and capturing the quality of user-generated multilingual structured data on the web. We also discuss ongoing editorial practices, which could encourage the use of higher-quality references in a more immediate way. All data and code used in the study are available on GitHub for feedback and further improvement and deployment by the research community.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。