技术报告：使用基于标记和基于标记的相似性进行实体提取
A Technical Report: Entity Extraction using Both Character-based and Token-based Similarity
摘要：
实体提取是许多文本挖掘任务的基础，例如组织名称识别。一种流行的实体提取方法是基于将文档中的候选子串与实体字典进行匹配。为了处理实体的拼写错误和名称变化，通常匹配是近似的，并且使用编辑或Jaccard距离来测量子串候选者和实体之间的不相似性。对于从自由文本中提取近似实体，现有的工作仅考虑基于字符或仅基于标记的相似性，因此不能同时处理标记级别和错别字的微小变化。在本文中，我们解决这个问题，同时考虑基于字符的相似性和基于标记的相似性（即两级相似性）。测量一级（例如基于字符的）相似性在计算上是昂贵的，并且测量两级相似性显著地更昂贵。通过利用两级相似性和令牌的权重的属性，我们开发了新的技术，以显着减少子串候选人的数量，需要计算两级相似性对字典的实体。在真实世界数据集上进行的实验表明，该算法能够有效地从文档中提取实体，并获得了[0.91，0.97]范围内的高F1值。
英文摘要：
Entity extraction is fundamental to many text mining tasks such as organisation name recognition. A popular approach to entity extraction is based on matching sub-string candidates in a document against a dictionary of entities. To handle spelling errors and name variations of entities, usually the matching is approximate and edit or Jaccard distance is used to measure dissimilarity between sub-string candidates and the entities. For approximate entity extraction from free text, existing work considers solely character-based or solely token-based similarity and hence cannot simultaneously deal with minor variations at token level and typos. In this paper, we address this problem by considering both character-based similarity and token-based similarity (i.e. two-level similarity). Measuring one-level (e.g. character-based) similarity is computationally expensive, and measuring two-level similarity is dramatically more expensive. By exploiting the properties of the two-level similarity and the weights of tokens, we develop novel techniques to significantly reduce the number of sub-string candidates that require computation of two-level similarity against the dictionary of entities. A comprehensive experimental study on real world datasets show that our algorithm can efficiently extract entities from documents and produce a high F1 score in the range of [0.91, 0.97].
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
