LLM驱动的多模态意见表达识别
LLM-Driven Multimodal Opinion Expression Identification
AI总结：
研究提出多模态意见表达识别（MOEI）任务及LLM驱动的STOEI方法，结合文本与语音模态，在构建的数据集上性能提升9.20%达SOTA。
摘要：
意见表达识别（OEI）在NLP中至关重要，应用范围从语音助手到抑郁症诊断。本研究将OEI扩展至多模态输入，强调听觉线索在传递文本无法企及的情感微妙之处方面的重要性。我们引入了新颖的多模态OEI（MOEI）任务，整合文本与语音以反映真实场景。利用CMU MOSEI和IEMOCAP数据集，我们构建了CI-MOEI数据集。此外，将文本转语音（TTS）技术应用于MPQA数据集，获得CIM-OEI数据集。我们为OEI任务设计了模板，以充分利用大型语言模型（LLMs）的生成能力。进一步推进，我们提出LLM驱动的方法STOEI，结合语音和文本模态来识别意见表达。实验表明，MOEI显著提升了性能，我们的方法优于现有方法9.20%并取得SOTA结果。
英文摘要：
Opinion Expression Identification (OEI) is essential in NLP for applications ranging from voice assistants to depression diagnosis. This study extends OEI to encompass multimodal inputs, underlining the significance of auditory cues in delivering emotional subtleties beyond the capabilities of text. We introduce a novel multimodal OEI (MOEI) task, integrating text and speech to mirror real-world scenarios. Utilizing CMU MOSEI and IEMOCAP datasets, we construct the CI-MOEI dataset. Additionally, Text-to-Speech (TTS) technology is applied to the MPQA dataset to obtain the CIM-OEI dataset. We design a template for the OEI task to take full advantage of the generative power of large language models (LLMs). Advancing further, we propose an LLM-driven method STOEI, which combines speech and text modal to identify opinion expressions. Our experiments demonstrate that MOEI significantly improves the performance while our method outperforms existing methods by 9.20\% and obtains SOTA results.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
