视觉落地对话中由话语感知理解引导的指称表达生成
Referring Expression Generation in Visually Grounded Dialogue with Discourse-aware Comprehension Guiding
AI总结：
针对视觉落地对话中的指称表达生成，本文提出两阶段方法，先用文本和视觉条件自回归生成候选 RE，再通过话语感知理解引导重排序，提升了表达的区分性与文本-图像检索准确率。
摘要：
我们提出了一种视觉落地对话中的指称表达生成（REG）方法，旨在生成既具有区分性又符合话语语境的指称表达（RE）。我们的方法包含两个阶段。首先，我们将 REG 建模为以文本和图像为条件的下一 token 预测任务。RE 根据其前文语言语境以及指称对象的视觉表征进行自回归生成。其次，我们提出将话语感知理解引导用于一种生成并重排序策略：由我们的 REG 模型生成的候选 RE，会依据其依赖话语的区分能力被重新排序。人工评估结果表明，我们提出的两阶段方法能够有效生成具有区分性的 RE；与使用贪心解码生成的 RE 相比，经重排序的 RE 在文本-图像检索准确率方面表现更高。
英文摘要：
We propose an approach to referring expression generation (REG) in visually grounded dialogue that is meant to produce referring expressions (REs) that are both discriminative and discourse-appropriate. Our method constitutes a two-stage process. First, we model REG as a text- and image-conditioned next-token prediction task. REs are autoregressively generated based on their preceding linguistic context and a visual representation of the referent. Second, we propose the use of discourse-aware comprehension guiding as part of a generate-and-rerank strategy through which candidate REs generated with our REG model are reranked based on their discourse-dependent discriminatory power. Results from our human evaluation indicate that our proposed two-stage approach is effective in producing discriminative REs, with higher performance in terms of text-image retrieval accuracy for reranked REs compared to those generated using greedy decoding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
