面向多模态命名实体识别与多模态关系抽取的思维链提示蒸馏
Chain-of-Thought Prompt Distillation for Multimodal Named Entity Recognition and Multimodal Relation Extraction
AI总结：
本研究针对多模态命名实体识别与关系抽取的推理需求，提出条件提示蒸馏方法，将大语言模型的思维链推理能力蒸馏至紧凑学生模型，在纯文本场景下也能实现SOTA准确率，兼具可解释性等多项优势。
摘要：
多模态命名实体识别（MNER）和多模态关系抽取（MRE）需要具备复杂语言理解与多模态理解的基础推理能力。本研究探索通过生成思维链（CoT，即一系列中间推理步骤），将大语言模型（LLM）的推理能力蒸馏到更紧凑的学生模型中。具体而言，我们首先通过覆盖多粒度（名词、句子、多模态）和数据增强（风格、实体、图像）维度的CoT提示，示例化从LLM中引出此类推理能力的过程。随后，我们提出一种新颖的条件提示蒸馏方法，以吸收LLM的常识推理能力，从而提升学生模型处理纯文本输入的效用，无需额外添加图像和CoT知识。大量实验表明，我们的方法在MNER和MRE数据集上取得了当前最优（SOTA）准确率，并且在可解释性、数据效率和跨域泛化性方面展现出诸多优势。
英文摘要：
Multimodal Named Entity Recognition (MNER) and Multimodal Relation Extraction (MRE) necessitate the fundamental reasoning capacity for intricate linguistic and multimodal comprehension. In this study, we explore distilling the reasoning ability of large language models (LLMs) into a more compact student model by generating a \textit{chain of thought} (CoT) -- a sequence of intermediate reasoning steps. Specifically, we commence by exemplifying the elicitation of such reasoning ability from LLMs through CoT prompts covering multi-grain (noun, sentence, multimodality) and data-augmentation (style, entity, image) dimensions. Subsequently, we present a novel conditional prompt distillation method to assimilate the commonsense reasoning ability from LLMs, thereby enhancing the utility of the student model in addressing text-only inputs without the requisite addition of image and CoT knowledge. Extensive experiments reveal that our approach attains state-of-the-art accuracy and manifests a plethora of advantages concerning interpretability, data efficiency, and cross-domain generalization on MNER and MRE datasets.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
