探索图像描述中的可供性与情境意义：多模态分析
Exploring Affordance and Situated Meaning in Image Captions: A Multimodal Analysis
AI总结：
本文从计算认知语言学视角，通过标注Flickr30k图像的五种感知属性并分析其与描述文本的关联，揭示了可供性与情境意义在多模态语义理解中的重要作用。
摘要：
本文从计算认知语言学的视角探讨了多模态语义表示的奠基问题。我们为 Flickr30k 数据集中的图像标注了五种感知属性：Affordance（可供性）、Perceptual Salience（感知显著性）、Object Number（对象数量）、Gaze Cueing（注视线索）和 Ecological Niche Association (ENA)（生态位关联），并考察了它们与图像描述中文本元素的关联。研究结果表明，与展示 telic affordance（目的性可供性）的图像相比，具有 Gibsonian affordance（吉布森可供性）的图像更频繁地出现包含“holding-verbs”（持握类动词）和“container-nouns”（容器类名词）的描述。Perceptual Salience、Object Number 和 ENA 也与语言表达的选择存在关联。本研究表明，对对象或事件的全面理解需要认知注意力、语言的语义细微差别以及跨多模态的整合。我们强调了情境意义和可供性奠基在自然语言理解中的至关重要性，并有望在多种场景中推进类人解释能力。
英文摘要：
This paper explores the grounding issue regarding multimodal semantic representation from a computational cognitive-linguistic view. We annotate images from the Flickr30k dataset with five perceptual properties: Affordance, Perceptual Salience, Object Number, Gaze Cueing, and Ecological Niche Association (ENA), and examine their association with textual elements in the image captions. Our findings reveal that images with Gibsonian affordance show a higher frequency of captions containing 'holding-verbs' and 'container-nouns' compared to images displaying telic affordance. Perceptual Salience, Object Number, and ENA are also associated with the choice of linguistic expressions. Our study demonstrates that comprehensive understanding of objects or events requires cognitive attention, semantic nuances in language, and integration across multiple modalities. We highlight the vital importance of situated meaning and affordance grounding in natural language understanding, with the potential to advance human-like interpretation in various scenarios.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
