使用自回归语言模型检测视觉基础对话中的指称表达
Detecting Referring Expressions in Visually Grounded Dialogue with Autoregressive Language Models
AI总结：
本文探索使用纯文本自回归大型语言模型，通过下一词元预测划定提及跨度边界，从视觉基础对话中提取指称表达，发现即使模型规模中等且数据集较小该方法依然有效，同时指出该任务本质是多模态问题。
摘要：
本文探索使用纯文本自回归语言建模方法，从视觉基础对话中提取指称表达。具体而言，旨在研究仅凭语言上下文能在多大程度上提示检测那些在对话视觉上下文中具有（视觉可感知）所指对象的提及。为此，我们适配预训练大型语言模型（LLM），通过在文本中利用下一词元预测划定提及跨度边界，对展开对话中的提及跨度执行相对粗粒度的标注。研究结果表明，即使使用中等规模的LLM、较小的数据集和参数高效微调，纯文本方法依然有效，突显了语言上下文对该任务的相对重要性。尽管如此，我们认为该任务本质上是一个多模态问题，并讨论了单模态方法的根本局限性。
英文摘要：
In this paper, we explore the use of a text-only, autoregressive language modeling approach for the extraction of referring expressions from visually grounded dialogue. More specifically, the aim is to investigate the extent to which the linguistic context alone can inform the detection of mentions that have a (visually perceivable) referent in the visual context of the conversation. To this end, we adapt a pretrained large language model (LLM) to perform a relatively course-grained annotation of mention spans in unfolding conversations by demarcating mention span boundaries in text via next-token prediction. Our findings indicate that even when using a moderately sized LLM, relatively small datasets, and parameter-efficient fine-tuning, a text-only approach can be effective, highlighting the relative importance of the linguistic context for this task. Nevertheless, we argue that the task represents an inherently multimodal problem and discuss limitations fundamental to unimodal approaches.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
