从大型语言模型中蒸馏濒危物种命名实体识别模型
Distilling Named Entity Recognition Models for Endangered Species from Large Language Models
AI总结：
该研究通过上下文学习从GPT-4蒸馏知识，构建了包含3.6K句子的濒危物种NER和RE数据集，并微调BERT模型，有效实现了濒危物种文本检测。
摘要：
自然语言处理（NLP）从业者正利用大型语言模型（LLM），从专利、论文和学位论文等半结构化和非结构化数据源中创建结构化数据集，而无需具备特定领域知识。同时，生态学专家正在寻找各种方法来保护生物多样性。为了助力这些努力，我们聚焦于濒危物种，并通过上下文学习从GPT-4中蒸馏知识。实际上，我们通过两阶段过程创建了用于命名实体识别（NER）和关系抽取（RE）的数据集：1）我们从GPT-4生成了四类濒危物种的合成数据；2）人工验证了合成数据的事实准确性，从而得到黄金数据。最终，我们的新数据集共包含3.6K个句子，平均分为1.8K个NER句子和1.8K个RE句子。然后，构建的数据集被用于微调通用BERT和特定领域的BERT变体，完成了从GPT-4到BERT的知识蒸馏过程，因为GPT-4资源消耗大。实验表明，我们的知识转移方法在创建适合从文本中检测濒危物种的NER模型方面是有效的。
英文摘要：
Natural language processing (NLP) practitioners are leveraging large language models (LLM) to create structured datasets from semi-structured and unstructured data sources such as patents, papers, and theses, without having domain-specific knowledge. At the same time, ecological experts are searching for a variety of means to preserve biodiversity. To contribute to these efforts, we focused on endangered species and through in-context learning, we distilled knowledge from GPT-4. In effect, we created datasets for both named entity recognition (NER) and relation extraction (RE) via a two-stage process: 1) we generated synthetic data from GPT-4 of four classes of endangered species, 2) humans verified the factual accuracy of the synthetic data, resulting in gold data. Eventually, our novel dataset contains a total of 3.6K sentences, evenly divided between 1.8K NER and 1.8K RE sentences. The constructed dataset was then used to fine-tune both general BERT and domain-specific BERT variants, completing the knowledge distillation process from GPT-4 to BERT, because GPT-4 is resource intensive. Experiments show that our knowledge transfer approach is effective at creating a NER model suitable for detecting endangered species from texts.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
