VLM-SlideEval：评估视觉语言模型在PPT中的结构化理解与扰动敏感性
VLM-SlideEval: Evaluating VLMs on Structured Comprehension and Perturbation Sensitivity in PPT
AI总结：
该论文提出VLM-SlideEval框架，从元素提取、扰动鲁棒性和叙事理解三方面评估视觉语言模型在PPT理解上的能力，发现其像素级提取不足且难以捕捉跨幻灯片结构，需引入评判者循环以改进智能体评估。
摘要：
视觉语言模型（VLMs）越来越多地被用于评估多模态内容，包括演示文稿幻灯片，然而尽管它们在智能体、模型前向流水线中作为评判者的作用日益增强，其对幻灯片的特定理解仍未得到充分探索。我们提出了VLM-SlideEval，一个评估框架，从三个维度探究视觉语言模型：（1）从幻灯片图像中提取与地面真值对齐的元素级信息；（2）对几何、样式和文本的受控扰动的鲁棒性；（3）更高级的理解，例如从打乱的幻灯片中恢复演示文稿的叙事顺序。利用Zenodo上公开可用的演示文稿（https://huggingface.co/datasets/Forceless/Zenodo10K/viewer/default/pptx），我们将来自PowerPoint XML和实时渲染的地面真值元素元数据标准化为统一、可验证的模式。实验上，视觉语言模型在像素级精确提取方面表现不佳，在受控扰动下显示出非平凡的协议、保真度和一致性，而在单张幻灯片内容理解上表现更好；然而，它们不能可靠地捕捉跨幻灯片的叙事结构。这些结果凸显了当前视觉语言模型在幻灯片评估中的局限性，并促使开发经过校准的、以评判者为中心的评估器，以驱动智能体流水线中的迭代优化和选择。
英文摘要：
Vision-language models (VLMs) are increasingly used to evaluate multimodal content, including presentation slides, yet their slide-specific understanding remains underexplored {despite their growing role as critics in agentic, model-forward pipelines}. We introduce VLM-SlideEval, an evaluation framework that probes VLMs along three axes: (1) element-level extraction from slide images aligned to ground truth; (2) robustness to controlled perturbations in geometry, style, and text; and (3) higher-level comprehension, such as recovering a deck's narrative order from shuffled slides. Using publicly available decks from Zenodo (https://huggingface.co/datasets/Forceless/Zenodo10K/viewer/default/pptx), we standardize ground-truth element metadata from PowerPoint XML and live renderings into a unified, verifiable schema. Empirically, VLMs underperform on pixel-accurate extraction and show non-trivial agreement, fidelity, and consistency under controlled perturbations, while performing better on single-slide content understanding; however, they do not reliably capture narrative structure across slides. These results highlight the limits of current VLMs for slide evaluation and motivate calibrated, critic-in-the-loop evaluators that drive iterative refinement and selection in agentic pipelines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
