一种多模态框架，用于将人类语言描述与视觉感知数据对齐
A Multimodal Framework for Aligning Human Linguistic Descriptions with Visual Perceptual Data
AI总结：
本文提出一种多模态框架，通过结合语言和视觉信息，实现了对人类指称行为的稳健建模，并在经典认知基准上取得了优于人类的表现。
摘要：
建立自然语言表达与视觉感知之间的稳定映射是认知科学和人工智能的基础问题。人类经常在嘈杂、模糊的感知环境中将语言参考固定化，但支持这种跨模态对齐的机制仍不为人所知。在本工作中，我们介绍了一个计算框架，旨在通过将语言陈述与从大规模众包图像中衍生出的感知表示相结合，来建模人类参照解释的核心方面。该系统通过结合尺度不变特征变换（SIFT）对齐与通用质量指数（UQI）来近似人类感知分类，以在认知合理的特征空间中量化相似性，同时一组语言预处理和查询转换操作捕捉了指称表达中的语用变化。我们对斯坦福重复参考游戏语料库（15,000条陈述配以积木刺激）进行了评估，这是一种专门设计用来探测人类水平感知模糊性和协调性的范式。我们的框架实现了稳健的指称固定。它比人类对话者少用了65%的陈述就能达到稳定的映射，并且能够从单个指称表达中正确识别目标对象的频率为41.66%（相比之下，人类为20%）。这些结果表明，相对简单的感知-语言对齐机制可以在经典认知基准上产生与人类竞争的行为，并为 grounded communication、感知推理和跨模态概念形成模型提供了见解。代码可在 https://anonymous.4open.science/r/metasequoia-9D13/README.md 上获得。
英文摘要：
Establishing stable mappings between natural language expressions and visual percepts is a foundational problem for both cognitive science and artificial intelligence. Humans routinely ground linguistic reference in noisy, ambiguous perceptual contexts, yet the mechanisms supporting such cross-modal alignment remain poorly understood. In this work, we introduce a computational framework designed to model core aspects of human referential interpretation by integrating linguistic utterances with perceptual representations derived from large-scale, crowd-sourced imagery. The system approximates human perceptual categorization by combining scale-invariant feature transform (SIFT) alignment with the Universal Quality Index (UQI) to quantify similarity in a cognitively plausible feature space, while a set of linguistic preprocessing and query-transformation operations captures pragmatic variability in referring expressions. We evaluate the model on the Stanford Repeated Reference Game corpus (15,000 utterances paired with tangram stimuli), a paradigm explicitly developed to probe human-level perceptual ambiguity and coordination. Our framework achieves robust referential grounding. It requires 65\% fewer utterances than human interlocutors to reach stable mappings and can correctly identify target objects from single referring expressions 41.66\% of the time (versus 20\% for humans).These results suggest that relatively simple perceptual-linguistic alignment mechanisms can yield human-competitive behavior on a classic cognitive benchmark, and offers insights into models of grounded communication, perceptual inference, and cross-modal concept formation. Code is available at https://anonymous.4open.science/r/metasequoia-9D13/README.md .
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
