真正的(视觉)谎言:分析生成式AI如何识别意图性、修辞和误导性在可视化谎言中
True (VIS) Lies: Analyzing How Generative AI Recognizes Intentionality, Rhetoric, and Misleadingness in Visualization Lies
AI总结:
本文研究多模态大语言模型识别和解释误导性可视化的能力,并通过2336条新冠相关推文数据集分析其识别原因和潜在意图性,评估16种前沿模型并对比人类专家行为。
摘要:
本研究探讨了多模态大语言模型识别和解释误导性可视化的能力,并通过2336条新冠相关推文数据集分析其识别原因和潜在意图性。我们的分析借鉴了可视化修辞学的概念和一个新开发的作者意图分类法作为解释视角。我们提出了三个研究问题,并通过包含一半存在误导性可视化的2336条新冠相关推文数据集进行实验性解答,并补充了来自VisLies的现实世界例子,该例子展示了感知、认知和概念错误。为了确保覆盖当前LLM的广泛范围,我们评估了16种前沿模型。其中15种是开源模型,涵盖了各种模型大小、架构家族和推理能力。所选模型包括小型模型,如Nemotron-Nano-V2-VL(12B参数)、Mistral-Small-3.2(24B)、DeepSeek-VL2(27B)、Gemma3(27B)和GTA1(32B);中型模型,如Qianfan-VL(70B)、Molmo(72B)、GLM-4.5V(108B)、LLaVA-NeXT(110B)和Pixtral-Large(124B);以及大型模型,如Qwen3-VL(235B)、InternVL3.5(241B)、Step3(321B)、Llama-4-Maverick(400B)和Kimi-K2.5(1000B)。此外,我们还采用了OpenAI GPT-5.4,一种前沿的专有模型。为了建立人类对这些任务的看法,我们还与可视化专家进行了用户研究,以评估人们如何感知修辞技巧和误导性可视化背后的作者意图。这使得可以比较模型和专家的行为,揭示相似性和差异性,从而提供关于LLM与人类判断一致和不一致之处的见解。
英文摘要:
This study investigates the ability of multimodal Large Language Models (LLMs) to identify and interpret misleading visualizations, and recognize these observations along with their underlying causes and potential intentionality. Our analysis leverages concepts from visualization rhetoric and a newly developed taxonomy of authorial intents as explanatory lenses. We formulated three research questions and addressed them experimentally using a dataset of 2,336 COVID-19-related tweets, half of which contain misleading visualizations, and supplemented it with real-world examples of perceptual, cognitive, and conceptual errors drawn from VisLies, the IEEE VIS community event dedicated to showcasing deceptive and misleading visualizations. To ensure broad coverage of the current LLM landscape, we evaluated 16 state-of-the-art models. Among them, 15 are open-weight models, spanning a wide range of model sizes, architectural families, and reasoning capabilities. The selection comprises small models, namely Nemotron-Nano-V2-VL (12B parameters), Mistral-Small-3.2 (24B), DeepSeek-VL2 (27B), Gemma3 (27B), and GTA1 (32B); medium-sized models, namely Qianfan-VL (70B), Molmo (72B), GLM-4.5V (108B), LLaVA-NeXT (110B), and Pixtral-Large (124B); and large models, namely Qwen3-VL (235B), InternVL3.5 (241B), Step3 (321B), Llama-4-Maverick (400B), and Kimi-K2.5 (1000B). In addition, we employed OpenAI GPT-5.4, a frontier proprietary model. To establish a human perspective on these tasks, we also conducted a user study with visualization experts to assess how people perceive rhetorical techniques and the authorial intentions behind the same misleading visualizations. This allows comparison between model and expert behavior, revealing similarities and differences that provide insights into where LLMs align with human judgment and where they diverge.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。