PosterSum：科学海报摘要的多模态基准
PosterSum: A Multimodal Benchmark for Scientific Poster Summarization
AI总结：
PosterSum基准包含16,305对科学海报与摘要，用于评估多模态模型；提出的Segment & Summarize层次化方法在ROUGE-L上提升3.14%，优于现有MLLMs。
摘要：
从多模态文档生成准确且简洁的文本摘要具有挑战性，尤其是在处理科学海报等视觉复杂内容时。我们引入了PosterSum，一个新颖的基准，旨在推动能够理解科学海报并将其总结为研究论文摘要的视觉语言模型的发展。我们的数据集包含16,305张会议海报，每张海报配有相应的摘要作为总结。每张海报以图像格式提供，并呈现多样的视觉理解挑战，如复杂布局、密集文本区域、表格和图形。我们在PosterSum上对最先进的多模态大语言模型（MLLMs）进行了基准测试，结果表明它们在准确解释和总结科学海报方面存在困难。我们提出了Segment & Summarize，一种层次化方法，在自动化指标上优于当前MLLMs，在ROUGE-L上获得了3.14%的提升。这将作为未来海报摘要研究的起点。
英文摘要：
Generating accurate and concise textual summaries from multimodal documents is challenging, especially when dealing with visually complex content like scientific posters. We introduce PosterSum, a novel benchmark to advance the development of vision-language models that can understand and summarize scientific posters into research paper abstracts. Our dataset contains 16,305 conference posters paired with their corresponding abstracts as summaries. Each poster is provided in image format and presents diverse visual understanding challenges, such as complex layouts, dense text regions, tables, and figures. We benchmark state-of-the-art Multimodal Large Language Models (MLLMs) on PosterSum and demonstrate that they struggle to accurately interpret and summarize scientific posters. We propose Segment & Summarize, a hierarchical method that outperforms current MLLMs on automated metrics, achieving a 3.14% gain in ROUGE-L. This will serve as a starting point for future research on poster summarization.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
