arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2304.04057cs.IR
收藏

不完美的自动标引如何影响语义搜索性能?

How Does Imperfect Automatic Indexing Affect Semantic Search Performance?

Mengtian Guo, David Gotz, Yue Wang

AI总结:

研究自动标引算法分配错误术语对布尔语义搜索性能的影响,通过MeSH术语和生物医学文献搜索案例发现概率逻辑优于布尔逻辑、查询性能受最差术语限制,且混合少量人工标引可恢复优异性能。

摘要:

健康领域的文档通常使用来自受控词表的语义概念(即术语)进行标注。随着这些文档数量变得庞大,标注工作越来越多地由算法完成。与人类相比,自动标引算法是不完美的,可能会给文档分配错误的术语,这会影响后续包含这些术语的搜索任务。在这项工作中,我们旨在理解在布尔语义搜索中使用不完美分配术语的性能影响。我们使用MeSH术语和生物医学文献搜索作为案例研究。我们在由MeSH术语组成的真实世界布尔查询上实现了多种自动标引算法,并发现:(1)概率逻辑比传统布尔逻辑能更好地处理不准确分配的术语;(2)查询级性能主要受限于查询中表现最差的术语;(3)将少量人工标引与自动标引混合可以恢复出色的查询级性能。这些发现为未来关于自动标引的工作提供了重要启示。

英文摘要:

Documents in the health domain are often annotated with semantic concepts (i.e., terms) from controlled vocabularies. As the volume of these documents gets large, the annotation work is increasingly done by algorithms. Compared to humans, automatic indexing algorithms are imperfect and may assign wrong terms to documents, which affect subsequent search tasks where queries contain these terms. In this work, we aim to understand the performance impact of using imperfectly assigned terms in Boolean semantic searches. We used MeSH terms and biomedical literature search as a case study. We implemented multiple automatic indexing algorithms on real-world Boolean queries that consist of MeSH terms, and found that (1) probabilistic logic can handle inaccurately assigned terms better than traditional Boolean logic, (2) query-level performance is mostly limited by lowest-performing terms in a query, and (3) mixing a small amount of human indexing with automatic indexing can regain excellent query-level performance. These findings provide important implications for future work on automatic indexing.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑