大型语言模型并非你所需要的全部
Large Language Models aren't all that you need
AI总结：
本文比较了传统CRF模型与微调LLM在多语言复杂命名实体识别任务上的表现，通过引入衰减辅助损失、三元组token混合和任务最优头部等创新技术，显著提升了LLM的宏观F1分数。
摘要：
本文描述了为解决SemEval 2023任务2：MultiCoNER II（多语言复杂命名实体识别）[1]而构建的架构和系统。我们评估了两种方法：(a)传统的条件随机场模型和(b)使用自定义头部微调的大型语言模型（LLM），并比较了这两种方法。探索的新颖想法是：1）衰减辅助损失（带残差）——我们在粗粒度NER的辅助任务上训练模型，并将此任务作为损失函数的一部分；2）三元组token混合——我们探索在最终NER层预测前混合相邻token嵌入的方法；3）任务最优头部——我们为LLM的最后一层探索了多种自定义头部和学习率。我们还探索了包括GPT-3在内的多个LLM，并尝试了多种dropout和其他超参数设置，最终模型在开发集上实现了0.85/0.84的微观和宏观F1值，在测试数据上实现了0.67/0.61。我们表明，虽然预训练LLM本身相比传统模型带来了分数的大幅提升，但通过用上述额外的特征/损失/模型工程技术增强LLM，可以对宏观F1分数做出实质性改进。
英文摘要：
This paper describes the architecture and systems built towards solving the SemEval 2023 Task 2: MultiCoNER II (Multilingual Complex Named Entity Recognition) [1]. We evaluate two approaches (a) a traditional Conditional Random Fields model and (b) a Large Language Model (LLM) fine-tuned with a customized head and compare the two approaches. The novel ideas explored are: 1) Decaying auxiliary loss (with residual) - where we train the model on an auxiliary task of Coarse-Grained NER and include this task as a part of the loss function 2) Triplet token blending - where we explore ways of blending the embeddings of neighboring tokens in the final NER layer prior to prediction 3) Task-optimal heads - where we explore a variety of custom heads and learning rates for the final layer of the LLM. We also explore multiple LLMs including GPT-3 and experiment with a variety of dropout and other hyperparameter settings before arriving at our final model which achieves micro & macro f1 of 0.85/0.84 (on dev) and 0.67/0.61 on the test data . We show that while pre-trained LLMs, by themselves, bring about a large improvement in scores as compared to traditional models, we also demonstrate that tangible improvements to the Macro-F1 score can be made by augmenting the LLM with additional feature/loss/model engineering techniques described above.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
