基于大型语言模型的社会媒体情境信号不确定性评估用于危机报道
LLM-based uncertainty assessment of social media situational signals for crisis reporting
AI总结：
本文提出一种考虑社会媒体声明可信度的自动化情境感知报告框架，通过评估外部代理数据下的置信度生成危机报告，提升危机响应中的信息优先级处理。
摘要：
社交媒体已成为灾难期间情境感知的关键来源，提供实时洞察于演变影响和新兴需求。为支持大规模危机响应，近期工作越来越多地利用大型语言模型（LLMs）自动分类和汇总社交媒体流中的情境信息。然而，现有方法隐式假设提取的情境声明等同可信，尽管危机发展过程中信息质量差异显著。本文提出一种不确定性感知的自动化情境感知报告框架，明确考虑社会媒体声明的可信度。首先，我们根据已建立的情境感知方案分类社交媒体帖子。其次，我们引入一个不确定性评估层，评估个体情境声明在外部代理数据条件下是否真实反映现实状况，同时明确模型的判断置信度。第三，我们利用这些不确定性评估生成危机报告，不仅传达报告内容，还说明报告的可信度。我们应用此框架处理超过200,000条与地震相关的Twitter/X帖子，使用USGS PAGER的影响摘要作为代表性的外部代理数据。我们主张明确表示不确定性有助于在时间压力下帮助人类危机沟通员优先处理信息，并提供一个将外部代理数据整合到LLM基于情境感知管道中的框架。
英文摘要：
Social media has become a critical source of situational awareness during disasters, providing real-time insights into evolving impacts and emerging needs. To support crisis response at scale, recent work has increasingly leveraged large language models (LLMs) to automatically classify and summarize situational information from social media streams. However, existing approaches implicitly assume that extracted situational claims are equally plausible, despite information quality varying substantially as a crisis unfolds. In this work, we propose an uncertainty-aware framework for automated situational awareness reporting that explicitly accounts for the plausibility of social media claims. First, we classify social media posts according to an established situational awareness schema. Second, we introduce an uncertainty assessment layer that evaluates whether individual situational claims plausibly reflect real-world conditions when conditioned on external proxy data, while explicitly eliciting the model's confidence in this judgment. Third, we use these uncertainty assessments to generate crisis reports that communicate not only what is being reported, but how certain those reports are. We apply this framework to over 200,000 earthquake-related Twitter/X posts, using impact summaries from the USGS PAGER as a representative external proxy. We argue that explicitly representing uncertainty supports human crisis communicators in prioritizing information under time pressure, and provides a framework for integrating external proxy data into LLM-based situational awareness pipelines.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
