HighlightBench：科学文档中基于标记的表格推理基准测试
HighlightBench: Benchmarking Markup-Driven Table Reasoning in Scientific Documents
AI总结：
本文提出HighlightBench，用于评估文档中基于标记的表格推理能力，通过五个任务家族分解评估，并提供可复现的基准流程，揭示模型在视觉提示与符号推理一致性下的稳定性问题。
摘要：
视觉标记如高亮、下划线和粗体文本在表格主导的文档中很常见。尽管多模态大语言模型在文档理解上取得了显著进展，但它们将这些提示作为显式逻辑指令的能力仍缺乏研究。更重要的是，现有评估无法区分模型是否未看到标记或未利用标记进行推理。这导致在评估表格上的标记条件行为时存在关键盲点。为解决这一差距，我们引入HighlightBench，一个用于评估基于标记的表格理解的诊断基准，将其评估分解为五个任务家族：标记定位、受限检索、局部关系、聚合与比较以及一致性和缺失性。我们进一步提供一个参考流程，使中间决策显式化，使基线可复现，并允许在感知到执行链中更细致地归因错误。实验显示，即使强大的模型在视觉提示必须与符号推理一致且在结构化输出约束下仍不稳定。
英文摘要：
Visual markups such as highlights, underlines, and bold text are common in table-centric documents. Although multimodal large language models (MLLMs) have made substantial progress in document understanding, their ability to treat such cues as explicit logical directives remains under-explored. More importantly, existing evaluations cannot distinguish whether a model fails to see the markup or fails to reason with it. This creates a key blind spot in assessing markup-conditioned behavior over tables. To address this gap, we introduce HighlightBench, a diagnostic benchmark for markup-driven table understanding that decomposes evaluation into five task families: Markup Grounding, Constrained Retrieval, Local Relations, Aggregation \& Comparison, and Consistency \& Missingness. We further provide a reference pipeline that makes intermediate decisions explicit, enabling reproducible baselines and finer-grained attribution of errors along the perception-to-execution chain. Experiments show that even strong models remain unstable when visual cues must be consistently aligned with symbolic reasoning under structured output constraints.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
