LLMs 能否感知危机的脉搏？评估孟加拉国七月起义中的危机情绪
Can LLMs Follow the Pulse of a Crisis? Evaluating Crisis Sentiment in Bangladesh's July Uprising
浏览论文内容
中文总结 AI 辅助
本文构建了孟加拉语危机情绪数据集 UNRESTSENT200K，包含约20万条起义期间评论，并评估多种模型，发现父帖上下文提升性能而时间偏移导致性能下降，为低资源危机情绪分析提供基准。
中文摘要 AI 辅助
危机情绪分析对于孟加拉语等低资源语言尤为具有挑战性，因为语言、语境和公众反应变化迅速。我们引入了 UNRESTSENT200K，一个孟加拉语危机情绪数据集，包含约 20 万条来自 2024 年 7 月至 8 月孟加拉国起义的 Facebook 和 YouTube 评论。该数据集涵盖五个与事件对齐的阶段，从早期升级和互联网中断到政权过渡以及后来的洪水危机。每条评论都与其父帖子相关联，从而可以在有或没有话语上下文的情况下进行评估。所有评论均通过完全人工流程进行标注，涉及 14 名以孟加拉语为母语的标注者和高级验证，达到了实质性一致性（kappa = 0.73，alpha = 0.71）和 94.2% 的盲审一致性。我们对微调编码器、提示 LLM 和 LoRA 微调 LLM 进行了基准测试。结果表明，父帖上下文始终能提升性能，而跨阶段的时间偏移会导致性能大幅下降。强大的 LLM 表现良好，但仍然难以处理讽刺、隐含的政治指涉和阶段依赖的含义。UNRESTSENT200K 为研究低资源危机话语中上下文感知和时间鲁棒的情绪分析提供了基准。UNRESTSENT200K 可在以下 https URL 获取。
英文摘要
Crisis sentiment analysis is especially challenging for low-resource languages such as Bangla, where language, context, and public reaction shift rapidly. We introduce UNRESTSENT200K, a Bangla crisis sentiment dataset with approximately 200K Facebook and YouTube comments from the July-August 2024 Bangladesh uprising. The dataset covers five event-aligned phases, from early escalation and internet blackout to regime transition and a later flood crisis. Each comment is linked to its parent post, enabling evaluation with and without discourse context. All comments are annotated through a fully human process involving 14 native Bangla-speaking annotators and senior validation, achieving substantial agreement (kappa = 0.73, alpha = 0.71) and 94.2% blind-audit agreement. We benchmark fine-tuned encoders, prompted LLMs, and LoRA-tuned LLMs. Results show that parent-post context consistently improves performance, while temporal shift across phases causes large performance drops. Strong LLMs perform well, but still struggle with sarcasm, implicit political references, and phase-dependent meaning. UNRESTSENT200K provides a benchmark for studying context-aware and temporally robust sentiment analysis in low-resource crisis discourse. UNRESTSENT200K is available at https://sami0055.github.io/UNRESTSENT200K/
发表机构
- North South University（南北大学）
- Charles Sturt University（查尔斯特大学）
机构由 AI 辅助整理，请以论文原文为准。
