RONA:基于连贯关系的实用多样化图像描述生成
RONA: Pragmatically Diverse Image Captioning with Coherence Relations
AI总结:
本文提出 RONA 提示策略,将连贯关系作为 MLLM 生成图像描述时的语用变化控制轴,从而在多个领域提升描述多样性及与真值的对齐程度。
摘要:
写作助手(如 Grammarly、Microsoft Copilot)传统上通过采用句法和语义变化来描述图像组成部分,从而生成多样化的图像描述。然而,人工撰写的描述更注重利用语用线索,在视觉描述之外传达核心信息。为提升描述多样性,必须探索将这些信息与视觉内容相结合进行传达的其他方式。我们提出 RONA,这是一种面向多模态大语言模型(MLLM)的新型提示策略,利用连贯关系作为语用变化的可控轴。我们证明,与跨多个领域的 MLLM 基线相比,RONA 生成的描述具有更好的整体多样性和真值对齐效果。我们的代码可在 https://github.com/aashish2000/RONA 获取。
英文摘要:
Writing Assistants (e.g., Grammarly, Microsoft Copilot) traditionally generate diverse image captions by employing syntactic and semantic variations to describe image components. However, human-written captions prioritize conveying a central message alongside visual descriptions using pragmatic cues. To enhance caption diversity, it is essential to explore alternative ways of communicating these messages in conjunction with visual content. We propose RONA, a novel prompting strategy for Multi-modal Large Language Models (MLLM) that leverages Coherence Relations as a controllable axis for pragmatic variations. We demonstrate that RONA generates captions with better overall diversity and ground-truth alignment, compared to MLLM baselines across multiple domains. Our code is available at: https://github.com/aashish2000/RONA
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。