面向图像叙述的半监督多模态共指消解
Semi-supervised multimodal coreference resolution in image narrations
AI总结：
针对图像叙述多模态共指消解中图文对齐、语言歧义与标注缺乏的挑战，提出数据高效的半监督跨模态方法，在共指与叙述歧义任务上优于强基线。
摘要：
本文研究多模态共指消解，特别是较长的描述性文本（即叙述）与图像配对的情况。由于细粒度的图文对齐、叙述语言固有的歧义性以及缺乏大型标注训练集，这带来了重大挑战。为应对这些挑战，我们提出了一种数据高效的方法，该方法在半监督设定下利用图像-叙述对来消解共指和叙述歧义。我们的方法在跨模态框架中为有标签和无标签数据设计了损失函数。评估表明，所提出的方法在共指消解和叙述歧义任务上，在定量和定性方面均优于强基线。
英文摘要：
In this paper, we study multimodal coreference resolution, specifically where a longer descriptive text, i.e., a narration is paired with an image. This poses significant challenges due to fine-grained image-text alignment, inherent ambiguity present in narrative language, and unavailability of large annotated training sets. To tackle these challenges, we present a data efficient semi-supervised approach that utilizes image-narration pairs to resolve coreferences and narrative grounding in a multimodal context. Our approach incorporates losses for both labeled and unlabeled data within a cross-modal framework. Our evaluation shows that the proposed approach outperforms strong baselines both quantitatively and qualitatively, for the tasks of coreference resolution and narrative grounding.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
