arXiv 2308.065522023-09-20cs.CL

MT4CrossOIE：面向跨语言开放信息抽取的多阶段调优

MT4CrossOIE: Multi-stage Tuning for Cross-lingual Open Information Extraction

Beijing Information Science and Technology University（北京信息科技大学）；Beihang University（北京航空航天大学）；The University of Hong Kong（香港大学）

Tongliang Li, Zixiang Wang, Linzheng Chai, Jian Yang, Jiaqi Bai, Yuwei Yin, Jiaheng Liu, Hongcheng Guo, Liqun Yang, Hebboul Zine el-abidine, Zhoujun Li

备注：: 10 pages

阅读 PDF

arXiv 页面 ↗HTML 原文 ↗

AI总结：

本文提出多阶段调优框架MT4CrossIE，通过注入语言特定模块并结合基于模型与数据的跨语言迁移，提升跨语言开放信息抽取性能。

摘要：

跨语言开放信息抽取旨在从多种语言的原始文本中提取结构化信息。以往工作使用共享的跨语言预训练模型来处理不同语言，但未能充分利用语言特定表示的潜力。本文提出了一种有效的多阶段调优框架MT4CrossIE，旨在通过向共享模型注入语言特定知识来增强跨语言开放信息抽取。具体而言，跨语言预训练模型首先在固定编码器中的共享语义空间（例如嵌入矩阵）上进行调优，然后在第二阶段优化其他组件。经过充分训练后，我们冻结预训练模型，并使用mixture-of-LoRAs调优多个额外的低秩语言特定模块，以实现基于模型的跨语言迁移。此外，我们利用两阶段提示来促使大语言模型（LLM）标注多语言原始数据，以实现基于数据的跨语言迁移。该模型在我们提出的数据集OpenIE4++上结合基于模型和基于数据的迁移技术，以多语言目标进行训练。在多个基准上的实验结果强调了聚合多个即插即用语言特定模块的重要性，并证明了MT4CrossIE在跨语言OIE中的有效性。

英文摘要：

Cross-lingual open information extraction aims to extract structured information from raw text across multiple languages. Previous work uses a shared cross-lingual pre-trained model to handle the different languages but underuses the potential of the language-specific representation. In this paper, we propose an effective multi-stage tuning framework called MT4CrossIE, designed for enhancing cross-lingual open information extraction by injecting language-specific knowledge into the shared model. Specifically, the cross-lingual pre-trained model is first tuned in a shared semantic space (e.g., embedding matrix) in the fixed encoder and then other components are optimized in the second stage. After enough training, we freeze the pre-trained model and tune the multiple extra low-rank language-specific modules using mixture-of-LoRAs for model-based cross-lingual transfer. In addition, we leverage two-stage prompting to encourage the large language model (LLM) to annotate the multi-lingual raw data for data-based cross-lingual transfer. The model is trained with multi-lingual objectives on our proposed dataset OpenIE4++ by combing the model-based and data-based transfer techniques. Experimental results on various benchmarks emphasize the importance of aggregating multiple plug-in-and-play language-specific modules and demonstrate the effectiveness of MT4CrossIE in cross-lingual OIE\footnote{\url{https://github.com/CSJianYang/Multilingual-Multimodal-NLP}}.

机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。