arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2405.12434cs.CL
收藏

面向自然语言推理的场景引导适配器消解词语模糊性研究

Resolving Word Vagueness with Scenario-guided Adapter for Natural Language Inference

Fudan University(复旦大学);University of Trento(特伦托大学)

Yonghao Liu, Mengyu Li, Di Liang, Ximing Li, Fausto Giunchiglia, Lan Huang, Xiaoyue Feng, Renchu Guan

AI总结:

针对传统NLI模型因缺乏情境视觉信息导致词语语义模糊、推理受限的问题,提出融合预训练语言知识与视觉信息的ScenaFuse适配器,通过交互与融合模块实现模态协同,经基准实验验证可有效提升NLI性能。

摘要:

自然语言推理(Natural Language Inference, NLI)是自然语言处理中的一项关键任务,旨在判定两个句子(通常称为前提句和假设句)之间的语义关系。然而,传统NLI模型仅依赖独立句子本身固有的语义信息,缺乏相关的情境视觉信息,由于语言的歧义性与模糊性,这会阻碍模型对句子 intended 含义的完整理解。为解决这一挑战,我们提出了一种创新的ScenaFuse适配器,该适配器可同时整合大规模预训练语言知识与相关视觉信息以应用于NLI任务。具体而言,我们首先设计了图像-句子交互模块,将视觉信息融入预训练模型的注意力机制中,使两种模态能够进行全面交互。此外,我们还引入了图像-句子融合模块,可自适应地整合来自图像的视觉信息与来自句子的语义信息。通过引入相关视觉信息并利用语言知识,我们的方法弥合了语言与视觉之间的鸿沟,提升了NLI任务中的理解与推理能力。大量基准实验表明,我们提出的ScenaFuse这一场景引导方法能够持续提升NLI性能。

英文摘要:

Natural Language Inference (NLI) is a crucial task in natural language processing that involves determining the relationship between two sentences, typically referred to as the premise and the hypothesis. However, traditional NLI models solely rely on the semantic information inherent in independent sentences and lack relevant situational visual information, which can hinder a complete understanding of the intended meaning of the sentences due to the ambiguity and vagueness of language. To address this challenge, we propose an innovative ScenaFuse adapter that simultaneously integrates large-scale pre-trained linguistic knowledge and relevant visual information for NLI tasks. Specifically, we first design an image-sentence interaction module to incorporate visuals into the attention mechanism of the pre-trained model, allowing the two modalities to interact comprehensively. Furthermore, we introduce an image-sentence fusion module that can adaptively integrate visual information from images and semantic information from sentences. By incorporating relevant visual information and leveraging linguistic knowledge, our approach bridges the gap between language and vision, leading to improved understanding and inference capabilities in NLI tasks. Extensive benchmark experiments demonstrate that our proposed ScenaFuse, a scenario-guided approach, consistently boosts NLI performance.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑