ToPro：面向跨语言序列标注任务的词元级提示分解
ToPro: Token-Level Prompt Decomposition for Cross-Lingual Sequence Labeling Tasks
AI总结：
本文提出词元级提示分解（ToPro）方法，将输入句子分解为单个词元并应用提示模板，在零样本跨语言NER和POS任务上优于现有微调方法，并达到最先进性能。
摘要：
基于提示的方法已成功应用于多语言预训练语言模型，用于零样本跨语言理解。然而，以往的研究大多聚焦于句子级分类任务，只有少数研究考虑了词元级标注任务，如命名实体识别（NER）和词性标注（POS）。在本文中，我们提出了词元级提示分解（ToPro），该方法促进了基于提示的方法在词元级序列标注任务中的应用。ToPro方法将输入句子分解为单个词元，并对每个词元应用一个提示模板。我们在多语言NER和POS标注数据集上的实验表明，基于ToPro的微调在零样本跨语言迁移中优于Vanilla微调和提示微调，尤其是对于与源语言英语类型学上不同的语言。当与mT5模型结合使用时，我们的方法还达到了最先进的性能。此外，我们在多语言大语言模型中的探索性研究表明，ToPro的表现远优于当前的上下文学习方法。总体而言，性能提升表明ToPro有可能作为一种新颖且简单的序列标注任务基准方法。
英文摘要：
Prompt-based methods have been successfully applied to multilingual pretrained language models for zero-shot cross-lingual understanding. However, most previous studies primarily focused on sentence-level classification tasks, and only a few considered token-level labeling tasks such as Named Entity Recognition (NER) and Part-of-Speech (POS) tagging. In this paper, we propose Token-Level Prompt Decomposition (ToPro), which facilitates the prompt-based method for token-level sequence labeling tasks. The ToPro method decomposes an input sentence into single tokens and applies one prompt template to each token. Our experiments on multilingual NER and POS tagging datasets demonstrate that ToPro-based fine-tuning outperforms Vanilla fine-tuning and Prompt-Tuning in zero-shot cross-lingual transfer, especially for languages that are typologically different from the source language English. Our method also attains state-of-the-art performance when employed with the mT5 model. Besides, our exploratory study in multilingual large language models shows that ToPro performs much better than the current in-context learning method. Overall, the performance improvements show that ToPro could potentially serve as a novel and simple benchmarking method for sequence labeling tasks.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
