Data annotation plays a crucial role in ensuring your named entity recognition (NER) projects are trained with the right information to learn from. Producing the most accurate labels is a challenge due to the complexity involved with annotation. Label inconsistency between multiple subsets of data annotation (e.g., training set and test set, or multiple training subsets) is an indicator of label mistakes. In this work, we present an empirical method to explore the relationship between label (in-)consistency and NER model performance. It can be used to validate the label consistency (or catches the inconsistency) in multiple sets of NER data annotation. In experiments, our method identified the label inconsistency of test data in SCIERC and CoNLL03 datasets (with 26.7% and 5.4% label mistakes). It validated the consistency in the corrected version of both datasets.
AdaK-NER: An Adaptive Top-K Approach for Named Entity Recognition with Incomplete Annotations
最先进的命名实体识别(NER)模型严重依赖于大量的完全注释的训练数据。然而,可访问的数据往往是不完整的注释,因为注释者通常缺乏全面的知识,在目标领域。通常,默认情况下,未注释的标记被视为非实体,而我们强调这些标记可以是非实体或任何实体的一部分。本文研究了不完全标注数据下的NER建模问题,在不完全标注数据下,只有一小部分命名实体被标注,而未标注的标记被每个可能的标记等价地多标注,考虑到多标记,大量的可能路径会分散训练模型对黄金路径(ground truth label序列)的注意力,从而影响学习能力。在本文中,我们提出了AdaK-NER,称为自适应top-K方法,以帮助模型专注于一个较小的可行区域,在那里黄金路径更有可能被定位。通过在英文和中文数据集上的大量实验,我们证明了该方法的优越性,在CoNLL-2003和两个中文数据集上的F值与现有的最先进的方法相比平均提高了2%和10%以上。
Adversarial Learning for Chinese NER from Crowd Annotations
School of Computer Science and Technology, Soochow University(苏州大学计算机科学与技术学院);School of Computer Science and Technology, Heilongjiang University(黑龙江大学计算机科学与技术学院);Alibaba Group(阿里巴巴集团);Shenzhen Gowild Robotics Co. Ltd(深圳狗尾草智能科技有限公司)
DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition
Seltasquare Seoul, Korea(首尔韩国Seltasquare);Seoul National University College of Medicine(首尔国立大学医学院);Biomedical Research Institute, Seoul National University Hospital(首尔国立大学医院生物医学研究所)
DeviceBERT: Applied Transfer Learning With Targeted Annotations and Vocabulary Enrichment to Identify Medical Device and Component Terminology in FDA Recall Summaries
En-ViMedNER: An English-Vietnamese Parallel Biomedical Corpus with UMLS Semantic Type Annotations
College of Engineering and Computer Science, VinUniversity(VinUniversity工程与计算机科学学院);Faculty of Engineering and IT, University of Technology Sydney(悉尼科技大学工程与信息技术学院);Center for AI Research, VinUniversity(VinUniversity人工智能研究中心);Monash University, Australia(莫纳什大学)
Partial Annotation Learning for Biomedical Entity Recognition
University of Chinese Academy of Sciences(中国科学院大学);National Science Library, Chinese Academy of Sciences(中国科学院国家科学图书馆);Institute for Logic, Language and Computation (ILLC), University of Amsterdam(阿姆斯特丹大学逻辑、语言与计算研究所)
Universal NER v2: Towards a Massively Multilingual Named Entity Recognition Benchmark
Northeastern University, USA(东北大学);Duolingo, USA(Duolingo);Comenius University Bratislava, Slovakia(布拉迪斯拉发康门尼大学);University of British Columbia, Canada(不列颠哥伦比亚大学);Alpinference, France(Alpinference);Research Institute for Artificial Intelligence, Romanian Academy, Romania(罗马尼亚科学院人工智能研究所);University of Ljubljana, Slovenia(卢布尔雅那大学);Aristotle University of Thessaloniki / ILSP, Athena Research Center, Greece(雅典纳基大学 / ILSP, 雅典研究中心);Universitas Pelita Harapan, Indonesia(佩利塔哈盼望大学);Seoul National University, South Korea(首尔国立大学);Independent Researcher, South Korea(韩国独立研究员);Democritus University of Thrace, Greece(特拉布宗大学);Ben-Gurion University of the Negev, Israel(贝尼·格里大学)
本文提出Universal NER v2,旨在构建多语言命名实体识别基准数据集,通过通用标签集和详尽标注指南实现标准化跨语言标注,提升多语言NER评估的准确性与一致性。
Human-Annotated NER Dataset for the Kyrgyz Language
The Cramer Project(Cramer项目);PDMI RAS, SPbU, KFU(俄罗斯科学院普列汉ov物理技术研究所、圣彼得堡大学、卡斯皮斯联邦大学);Dep. of Computer Linguistics(计算机语言学系);Information Technology Institute(信息技术研究所);St. Petersburg Dep. of Steklov Math. Institute(圣彼得堡斯捷克洛夫数学研究所部门);St. Petersburg State University(圣彼得堡国立大学)
本文构建了首个吉尔吉斯语人工标注命名实体识别数据集 KyrgyzNER,并通过多模型评估表明多语言预训练模型在低资源语言 NER 任务中具有良好潜力。
GLiNER-BioMed: A Suite of Efficient Models for Open Biomedical Named Entity Recognition
Department of Radiology and Medical Informatics, Faculty of Medicine, University of Geneva(日内瓦大学医学院放射学与医学信息学系);Knowledgator Engineering(Knowledgator工程)
Neural Modeling for Named Entities and Morphology (NEMO^2)
Bar Ilan University(巴伊兰大学);Open Media and Information Lab (OMILab), The Open University of Israel(以色列开放大学开放媒体与信息实验室);The Open University of Israel(以色列开放大学)
An analysis of full-size Russian complexly NER labelled corpus of Internet user reviews on the drugs based on deep learning and language neural nets
NRC "Kurchatov institute"(国家研究中心“库尔恰托夫研究所”);MEPhI National Research Nuclear University(莫斯科工程物理学院国家核研究大学);I.M. Sechenov First Moscow State Medical University (Sechenov University)(谢切诺夫第一莫斯科国立医科大学(谢切诺夫大学))
ReFineG: Synergizing Small Supervised Models and LLMs for Low-Resource Grounded Multimodal NER
School of Artificial Intelligence, Sun Yat-sen University(中山大学人工智能学院);Key Laboratory of Sustainable Tourism Smart Assessment Technology, Ministry of Culture and Tourism, Sun Yat-sen University(文化旅游可持续评估技术重点实验室,中华人民共和国文化和旅游部,中山大学);Beijing Normal University(北京师范大学);Institute of Software, Chinese Academy of Sciences(中国科学院软件研究所)