FloodBrain:基于大语言模型的网页检索增强生成式洪灾报告生成
FloodBrain: Flood Disaster Reporting by Web-based Retrieval Augmented Generation with an LLM
AI总结:
针对大语言模型易生成幻觉信息的问题,研究提出搭载复杂流水线的洪灾报告工具FloodBrain,通过网页检索增强生成准确洪灾报告,验证了GPT-4评分与人类评估的相关性,助力提升灾害救援协调效率。
摘要:
快速的灾害影响报告对于规划人道主义援助至关重要。大语言模型(LLM)以其撰写连贯文本、完成多种与影响报告相关任务(如问答或文本摘要)的能力而闻名。然而,LLM受限于其训练数据内的知识,且容易生成不准确的、即“幻觉”信息。为解决这一问题,我们推出了工具FloodBrain(floodbrain.com)所搭载的复杂流水线,该工具专门通过从网页中提取并整理信息来生成洪灾影响报告。我们的流水线整合网页搜索结果中的信息,以生成关于洪水事件的详尽且准确的报告。我们测试了不同LLM作为该工具的骨干模型,并在多项指标上将其生成的报告与人工撰写的报告进行对比。与其他研究类似,我们发现,在对比生成报告与人工撰写报告时,GPT-4给出的评分与人类评估者给出的评分存在显著相关性。此外,我们开展了消融实验,测试流水线各独立组件及其对最终报告的作用。我们希望通过该工具推动LLM在灾害影响报告领域的应用,缩短洪灾发生后人道主义救援工作的协调时间。
英文摘要:
Fast disaster impact reporting is crucial in planning humanitarian assistance. Large Language Models (LLMs) are well known for their ability to write coherent text and fulfill a variety of tasks relevant to impact reporting, such as question answering or text summarization. However, LLMs are constrained by the knowledge within their training data and are prone to generating inaccurate, or "hallucinated", information. To address this, we introduce a sophisticated pipeline embodied in our tool FloodBrain (floodbrain.com), specialized in generating flood disaster impact reports by extracting and curating information from the web. Our pipeline assimilates information from web search results to produce detailed and accurate reports on flood events. We test different LLMs as backbones in our tool and compare their generated reports to human-written reports on different metrics. Similar to other studies, we find a notable correlation between the scores assigned by GPT-4 and the scores given by human evaluators when comparing our generated reports to human-authored ones. Additionally, we conduct an ablation study to test our single pipeline components and their relevancy for the final reports. With our tool, we aim to advance the use of LLMs for disaster impact reporting and reduce the time for coordination of humanitarian efforts in the wake of flood disasters.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。