Entity matching (EM) is the problem of determining whether two records refer to same real-world entity, which is crucial in data integration, e.g., for product catalogs or address databases. A major drawback of many EM approaches is their dependence on labelled examples. We thus focus on the challenging setting of zero-shot entity matching where no labelled examples are available for an unseen target dataset. Recently, large language models (LLMs) have shown promising results for zero-shot EM, but their low throughput and high deployment cost limit their applicability and scalability.
We revisit the zero-shot EM problem with AnyMatch, a small language model fine-tuned in a transfer learning setup. We propose several novel data selection techniques to generate fine-tuning data for our model, e.g., by selecting difficult pairs to match via an AutoML filter, by generating additional attribute-level examples, and by controlling label imbalance in the data.
We conduct an extensive evaluation of the prediction quality and deployment cost of our model, in a comparison to thirteen baselines on nine benchmark datasets. We find that AnyMatch provides competitive prediction quality despite its small parameter size: it achieves the second-highest F1 score overall, and outperforms several other approaches that employ models with hundreds of billions of parameters. Furthermore, our approach exhibits major cost benefits: the average prediction quality of AnyMatch is within 4.4% of the state-of-the-art method MatchGPT with the proprietary trillion-parameter model GPT-4, yet AnyMatch requires four orders of magnitude less parameters and incurs a 3,899 times lower inference cost (in dollars per 1,000 tokens).
Entity Decomposition with Filtering: A Zero-Shot Clinical Named Entity Recognition Framework
The Ohio State University（俄亥俄州立大学）；Translational Data Analytics Institute, The Ohio State University（俄亥俄州立大学转化数据分析研究所）；College of Pharmacy, The Ohio State University（俄亥俄州立大学药学院）
Tsinghua University（清华大学）；Alibaba Group（阿里巴巴集团）；Seoul National University（首尔国立大学）
As an effective approach to tune pre-trained language models (PLMs) for specific tasks, prompt-learning has recently attracted much attention from researchers. By using \textit{cloze}-style language prompts to stimulate the versatile knowledge of PLMs, prompt-learning can achieve promising results on a series of NLP tasks, such as natural language inference, sentiment classification, and knowledge probing. In this work, we investigate the application of prompt-learning on fine-grained entity typing in fully supervised, few-shot and zero-shot scenarios. We first develop a simple and effective p…
ReverseNER: A Self-Generated Example-Driven Framework for Zero-Shot Named Entity Recognition with Large Language Models
School of Software Engineering, Xi’an Jiaotong University（西安交通大学软件学院）；China Mobile System Integration Co.,Ltd.（中移系统集成有限公司）；China Mobile Communications Group Co.,Ltd.（中国移动通信集团有限公司）
Neural document retrievers, including dense passage retrieval (DPR), have outperformed classical lexical-matching retrievers, such as BM25, when fine-tuned and tested on specific question-answering datasets. However, it has been shown that the existing dense retrievers do not generalize well not only out of domain but even in domain such as Wikipedia, especially when a named entity in a question is a dominant clue for retrieval. In this paper, we propose an approach toward in-domain generalization using the embeddings generated by the frozen language model trained with the entities in the doma…
Recently, prompt-based learning for pre-trained language models has succeeded in few-shot Named Entity Recognition (NER) by exploiting prompts as task guidance to increase label efficiency. However, previous prompt-based methods for few-shot NER have limitations such as a higher computational complexity, poor zero-shot ability, requiring manual prompt engineering, or lack of prompt robustness. In this work, we address these shortcomings by proposing a new prompt-based learning NER method with Question Answering (QA), called QaNER. Our approach includes 1) a refined strategy for converting NER …
University of Illinois at Urbana-Champaign（伊利诺伊大学厄巴纳-香槟分校）
We study the problem of few-shot Fine-grained Entity Typing (FET), where only a few annotated entity mentions with contexts are given for each entity type. Recently, prompt-based tuning has demonstrated superior performance to standard fine-tuning in few-shot scenarios by formulating the entity type classification task as a ''fill-in-the-blank'' problem. This allows effective utilization of the strong language modeling capability of Pre-trained Language Models (PLMs). Despite the success of current prompt-based tuning approaches, two major challenges remain: (1) the verbalizer in prompts is ei…
East China Normal University（华东师范大学）；Microsoft（微软）
Zero-shot relation triplet extraction (ZeroRTE) aims to extract relation triplets from unstructured texts under the zero-shot setting, where the relation sets at the training and testing stages are disjoint. Previous state-of-the-art method handles this challenging task by leveraging pretrained language models to generate data as additional training samples, which increases the training cost and severely constrains the model performance. To address the above issues, we propose a novel method named PCRED for ZeroRTE with Potential Candidate Relation Selection and Entity Boundary Detection. The …
DiZiNER: Disagreement-guided Instruction Refinement via Pilot Annotation Simulation for Zero-shot Named Entity Recognition
Seltasquare Seoul, Korea（首尔韩国Seltasquare）；Seoul National University College of Medicine（首尔国立大学医学院）；Biomedical Research Institute, Seoul National University Hospital（首尔国立大学医院生物医学研究所）
Transferring knowledge from one domain to another is of practical importance for many tasks in natural language processing, especially when the amount of available data in the target domain is limited. In this work, we propose a novel few-shot approach to domain adaptation in the context of Named Entity Recognition (NER). We propose a two-step approach consisting of a variable base module and a template module that leverages the knowledge captured in pre-trained language models with the help of simple descriptive patterns. Our approach is simple yet versatile and can be applied in few-shot and…
Zero-shot entity retrieval, aiming to link mentions to candidate entities under the zero-shot setting, is vital for many tasks in Natural Language Processing. Most existing methods represent mentions/entities via the sentence embeddings of corresponding context from the Pre-trained Language Model. However, we argue that such coarse-grained sentence embeddings can not fully model the mentions/entities, especially when the attention scores towards mentions/entities are relatively low. In this work, we propose GER, a \textbf{G}raph enhanced \textbf{E}ntity \textbf{R}etrieval framework, to capture…
Retrieval augmented generation based dynamic prompting for few-shot biomedical named entity recognition using large language models
Department of Biomedical Informatics, School of Medicine, Emory University（生物医学信息学系，医学院，埃默里大学）；Department of Computer Science, Emory University（计算机科学系，埃默里大学）；National Library of Medicine, National Institutes of Health（国家医学图书馆，国立卫生研究院）；Department of Biomedical Engineering, Georgia Institute of Technology and Emory University（生物医学工程系，佐治亚理工学院和埃默里大学）
Zero- and Few-shot Named Entity Recognition and Text Expansion in Medication Prescriptions using ChatGPT
Chiang Mai University（清迈大学）；Friedrich-Alexander-Universität Erlangen-Nürnberg（埃尔朗根-纽伦堡弗里德里希-亚历山大大学）；Universitätsklinikum Erlangen（埃尔朗根大学医院）；Center of Data Analytics and Knowledge Synthesis for Health Care（医疗保健数据分析与知识综合中心）；Medical University of Graz（格拉茨医科大学）
Beyond the Basics: Leveraging Large Language Model for Fine-Grained Medical Entity Recognition
Western Sydney University（西悉尼大学）；South Western Emergency Research Institute（西南急救研究所）；Garvan Institute of Medical Research（加文医学研究所以）；University of New South Wales（新南威尔士大学）；Liverpool Hospital（利物浦医院）
llmNER: (Zero|Few)-Shot Named Entity Recognition, Exploiting the Power of Large Language Models
University of Chile（智利大学）；Millennium Institute for Foundational Research on Data（数据基础研究千年研究所）；Catholic University of Chile（智利天主教大学）；Faculty of Physical and Mathematical Sciences, University of Chile（智利大学物理与数学学院）
本文提出 llmNER 库，通过简化提示构建与结果解析，支持基于大型语言模型的零样本和少样本命名实体识别，并在两个 NER 任务上验证其灵活性与有效性。
Fudan University（复旦大学）；Fudan-Aishu Cognitive Intelligence Joint Research Center（复旦-爱数认知智能联合研究中心）
Entity typing aims to assign types to the entity mentions in given texts. The traditional classification-based entity typing paradigm has two unignorable drawbacks: 1) it fails to assign an entity to the types beyond the predefined type set, and 2) it can hardly handle few-shot and zero-shot situations where many long-tail types only have few or even no training instances. To overcome these drawbacks, we propose a novel generative entity typing (GET) paradigm: given a text with an entity mention, the multiple types for the role that the entity plays in the text are generated with a pre-trained…
Self-Improving for Zero-Shot Named Entity Recognition with Large Language Models
College of Computer Science and Technology, Zhejiang University（浙江大学计算机科学与技术学院）；ZJU-UIUC Institute, Zhejiang University（浙江大学ZJU-UIUC学院）；National University of Singapore（新加坡国立大学）
International Institute of Information Technology, Bangalore（班加罗尔国际信息技术学院）；Indraprastha Institute of Information Technology, Delhi（德里印德拉普拉斯塔信息技术学院）；International Institute of Information Technology, Hyderabad（海得拉巴国际信息技术学院）
Online news and social media have been the de facto mediums to disseminate information globally from the beginning of the last decade. However, bias in content and purpose of intentions are not regulated, and managing bias is the responsibility of content consumers. In this regard, understanding the stances and biases of news sources towards specific entities becomes important. To address this problem, we use pretrained language models, which have been shown to bring about good results with no task-specific training or few-shot training. In this work, we approach the problem of characterizing …
Benchmarking Open-Source Large Language Models for Persian in Zero-Shot and Few-Shot Learning
Department of Medical Engineering（医学工程系）；Electrical Engineering Department（电气工程系）；Sharif University of Technology（谢里夫技术大学）；Department of Electrical Engineering（电气工程系）；University of Tehran（德黑兰大学）
Despite impressive results of language models for named entity recognition (NER), their generalization to varied textual genres, a growing entity type set, and new entities remains a challenge. Collecting thousands of annotations in each new case for training or fine-tuning is expensive and time-consuming. In contrast, humans can easily identify named entities given some simple instructions. Inspired by this, we challenge the reliance on large datasets and study pre-trained language models for NER in a meta-learning setup. First, we test named entity typing (NET) in a zero-shot transfer scenar…
Hefei University of Technology（合肥工业大学）；Umeå University（于默奥大学）
While Named Entity Recognition (NER) is a widely studied task, making inferences of entities with only a few labeled data has been challenging, especially for entities with nested structures. Unlike flat entities, entities and their nested entities are more likely to have similar semantic feature representations, drastically increasing difficulties in classifying different entity categories in the few-shot setting. Although prior work has briefly discussed nested structures in the context of few-shot learning, to our best knowledge, this paper is the first one specifically dedicated to studyin…
Medical entity retrieval is an integral component for understanding and communicating information across various health systems. Current approaches tend to work well on specific medical domains but generalize poorly to unseen sub-specialties. This is of increasing concern under a public health crisis as new medical conditions and drug treatments come to light frequently. Zero-shot retrieval is challenging due to the high degree of ambiguity and variability in medical corpora, making it difficult to build an accurate similarity measure between mentions and concepts. Medical knowledge graphs (KG…
Fine-tuning pre-trained language models has recently become a common practice in building NLP models for various tasks, especially few-shot tasks. We argue that under the few-shot setting, formulating fine-tuning closer to the pre-training objectives shall be able to unleash more benefits from the pre-trained language models. In this work, we take few-shot named entity recognition (NER) for a pilot study, where existing fine-tuning strategies are much different from pre-training. We propose a novel few-shot fine-tuning framework for NER, FFF-NER. Specifically, we introduce three new types of t…
Harbin Institute of Technology（哈尔滨工业大学）；Microsoft Research Asia（微软亚洲研究院）
Few-shot named entity recognition (NER) systems aim at recognizing novel-class named entities based on only a few labeled examples. In this paper, we present a decomposed meta-learning approach which addresses the problem of few-shot NER by sequentially tackling few-shot span detection and few-shot entity typing using meta-learning. In particular, we take the few-shot span detection as a sequence labeling problem and train the span detector by introducing the model-agnostic meta-learning (MAML) algorithm to find a good model parameter initialization that could fast adapt to new entity classes.…
Recently, considerable literature has grown up around the theme of few-shot named entity recognition (NER), but little published benchmark data specifically focused on the practical and challenging task. Current approaches collect existing supervised NER datasets and re-organize them to the few-shot setting for empirical study. These strategies conventionally aim to recognize coarse-grained entity types with few examples, while in practice, most unseen entity types are fine-grained. In this paper, we present Few-NERD, a large-scale human-annotated few-shot NER dataset with a hierarchy of 8 coa…
