CrisiText: 一个用于LLM训练的紧急通讯警示信息数据集
CrisiText: A dataset of warning messages for LLM training in emergency communication
AI总结：
CrisiText是首个大规模危机警示信息数据集，通过生成13种危机场景下的警示信息，提升LLM在紧急通讯中的训练效果。
摘要：
在危机情况下，如自然灾害或暴力攻击中有效识别威胁并减轻其潜在损害，对于保护受威胁个体至关重要。为解决这些挑战，人工智能已被用于帮助人类在紧急情况中。然而，NLP技术的使用仍然有限，主要集中在分类任务上。然而，使用NLG架构进行及时警示信息生成的显著潜力却一直被忽视。在本文中，我们提出了CrisiText，这是首个大规模数据集，用于生成13种不同危机场景下的警示信息。该数据集包含超过400,000条警示信息（涵盖近18,000种危机情境），旨在在事件发生期间和之后帮助平民。为了生成该数据集，我们从现有的危机描述开始，创建与场景相关的事件链。每个事件随后都配对一条警示信息。生成过程遵循专家撰写的指南，以确保术语的正确性和建议的真实性。此外，每条信息都附带三种次优警示类型，以允许研究不同的NLG方法。为此，我们进行了一系列实验，比较了监督微调设置与偏好对齐、零样本和少样本方法。我们还评估了模型在分布外场景中的性能，并评估了自动后编辑器的有效性。
英文摘要：
Effectively identifying threats and mitigating their potential damage during crisis situations, such as natural disasters or violent attacks, is paramount for safeguarding endangered individuals. To tackle these challenges, AI has been used in assisting humans in emergency situations. Still, the use of NLP techniques remains limited and mostly focuses on classification tasks. The significant potential of timely warning message generation using NLG architectures, however, has been largely overlooked. In this paper we present CrisiText, the first large-scale dataset for the generation of warning messages across 13 different types of crisis scenarios. The dataset contains more than 400,000 warning messages (spanning almost 18,000 crisis situations) aimed at assisting civilians during and after such events. To generate the dataset, we started from existing crisis descriptions and created chains of events related to the scenarios. Each event was then paired with a warning message. The generations follow experts' written guidelines to ensure correct terminology and factuality of their suggestions. Additionally, each message is accompanied by three suboptimal warning types to allow for the study of different NLG approaches. To this end, we conducted a series of experiments comparing supervised fine-tuning setups with preference alignment, zero-shot, and few-shot approaches. We further assessed model performance in out-of-distribution scenarios and evaluated the effectiveness of an automatic post-editor.
机构、总结、中文摘要均由AI分析而来，可能存在错误，请以论文正文为准。
