谁来评判框架？审计多模态大语言模型在事件级视角下对新闻框架的评判
Who Judges the Frame? Auditing Multimodal LLM Judges for News Framing Across Event-Level Perspectives
浏览论文内容
中文总结 AI 辅助
本研究审计多模态LLM作为新闻框架分析工具，通过事件级数据集和嵌入相似性度量，揭示模态效应、元数据敏感性与提示伪影，提出审计协议以提升媒体分析可靠性。
中文摘要 AI 辅助
重大世界事件的新闻报道不仅受报道内容的影响，还受事件如何通过文本、图像及其组合进行框架构建的影响。同时，包括多模态大语言模型（LLMs）在内的大语言模型，越来越多地被用作可扩展的工具，用于分析多模态媒体数据集中的框架、情感、意识形态倾向和视角差异。这带来了方法论上的挑战。当用作测量工具时，LLM输出可能不仅反映内容属性，还可能反映模型特定倾向、提示设计选择以及社会、政治、文化、语言或模态特定的假设。本研究审计了LLMs作为大规模多模态新闻报道中框架和视角分析工具的有效性。利用一个以事件为中心的2025-2026年新闻报道数据集，其中每个事件包含关于同一标题的左倾、中间和右倾文章，我们结合基于嵌入的事件内视角相似性度量与基于模型的框架构念评估，在跨模态特定和元数据可见的输入条件下进行。我们并不将数据集标签或模型输出视为地面真值，而是旨在检验基于LLM的媒体分析的有用性和局限性。本研究贡献了一个审计协议，强调在报告关于新闻框架和意识形态视角差异的实质性主张时，需要报告模态效应、元数据敏感性和提示引发的伪影。
英文摘要
News coverage of major world events is shaped not only by what is reported, but also by how events are framed through text, images and their combination. At the same time, Large Language Models (LLMs), including multimodal LLMs, are increasingly used as scalable instruments for analysing framing, sentiment, ideological slant and perspective differences in multimodal media datasets. This creates a methodological challenge. When used as measurement instruments, LLM outputs may reflect not only content properties, but also model-specific tendencies, prompt design choices, and social, political, cultural, linguistic or modality-specific assumptions. This work audits LLMs as instruments for large-scale framing and perspective analysis in multimodal news coverage. Using an event-centered dataset of 2025--2026 news coverage, where each event includes left-, center- and right-oriented articles about the same headline, we combine embedding-based measures of within-event viewpoint similarity with model-based assessments of framing constructs across modality-specific and metadata-visible input conditions. Rather than treating either dataset labels or model outputs as ground truth, our goal is to examine the usefulness and limitations of LLM-based media analysis. The study contributes an audit protocol that highlights the need to report modality effects, metadata sensitivity, and prompt-induced artifacts alongside substantive claims about news framing and ideological viewpoint differences.
发表机构
- Johannes Kepler University Linz（林茨约翰·开普勒大学）
- CONICET-UNCPBA, ISISTAN（阿根廷国家科学研究和技术委员会-内乌肯国立大学，ISISTAN研究所）
- Linz Institute of Technology（林茨技术学院）
机构由 AI 辅助整理，请以论文原文为准。
