面向科学传播中图表-图像连贯性锚定的多模态推理类型学
A Multimodal Reasoning Typology for Grounding Chart-Image Coherence in Science Communication
AI总结：
本研究提出R1-R5多模态推理类型学，刻画科学文献中图表、图像与文本的协同推理缺口，可系统识别连贯性锚定状态，缩小不同人群解读科学结论的认知差距。
摘要：
图表与图像在各类科学出版物中同步出现，但多数计算领域研究并未对二者的连贯性进行表征。本文认为，一张图表、其配套图像以及关联二者的说明文字共同构成一个多模态单元，读者解读该单元所需的推理工作存在系统性差异。为捕捉这种差异，本文构建了推理缺口类型学R1至R5，用于表征图表、图像与文本如何共同传递科学主张，以及该过程对读者提出的解读要求。部分图表-图像对仅对同一数据进行重述，其余场景下图表可用于量化图像定位的结构、将图像内容映射到外部变量、核验基于图像的主张，或共同构建任一单图板块都无法独立确立的分析框架。该类型学锚定传播锚定理论，由一名神经科学专家通过自底向上的方式，从包含79篇创伤性脑损伤论文、32组图表-图像对的语料库中推导得出。至关重要的是，该分级体系提供了识别锚定成功或失效节点的系统性机制，无需依赖主观推断。本文通过一项研究验证了该特性：由一名领域专家与三名非专家对25组图表-图像对的视觉语言模型（VLM）描述进行评判，结果显示该分级体系可预判评判结果的一致与分歧位置，分离出由上下文知识而非图像本身承载连贯性的节点。该类型学因此为图设计者提供了平衡文本与图表-图像对的系统化方法，可弥合专家与非专家群体解读科学结论的认知鸿沟。
英文摘要：
Charts and images appear together throughout scientific publications, yet most computational work does not characterize their coherence. We argue that a chart, its accompanying image, and the caption that links them form a multimodal unit, and that the inferential work required to read it varies systematically. To capture this variation, we develop a typology of reasoning gaps, R1 through R5, that characterizes how chart, image, and text jointly convey a scientific claim, and the interpretive work this demands of the reader. Some pairs restate the same data, while in other pairs, charts are used to quantify a structure the image localizes, project image content onto an external variable, audit an image-based claim, or jointly construct a frame that neither panel can establish alone. The typology is anchored in the grounding theory of communication and was derived bottom-up, with a neuroscience expert, from a corpus of 79 traumatic brain injury papers and 32 chart-image pairs. Crucially, the levels provide a systematic mechanism for identifying where grounding succeeds or breaks down, rather than leaving it to subjective inference. We show this in a study in which a domain expert and three non-experts judge vision-language model (VLM) descriptions of 25 pairs: the level predicts where their judgments align and where they diverge, isolating the points at which contextual knowledge, not the figure, carries coherence. This typology thus offers figure designers a systematic way to balance text against chart-image pairs, bridging the expert-to-non-expert divide in reading a scientific takeaway.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
