OpenEP:开放式未来事件预测
OpenEP: Open-Ended Future Event Prediction
AI总结:
针对现有未来事件预测任务结果范围固定的局限,提出开放式预测任务OpenEP,构建多视角的OpenEPBench数据集,提出利益相关者增强框架StkFEP,实验表明现有LLM在该开放式场景下预测仍具挑战。
摘要:
未来事件预测(FEP)是一项长期存在且至关重要的全球性任务,因为理解事件的演化能够助力早期风险识别、明智决策和战略规划。现有研究通常将事件预测视为分类任务,并将未来事件的结果限定在固定范围内,例如是/否问题、候选集和分类体系,这难以涵盖未来事件的所有可能结果。本文提出了OpenEP(开放式未来事件预测任务),该任务能够生成符合现实场景的灵活多样的预测结果,这主要体现在两个方面:其一,预测问题具有多样性,覆盖事件发展的不同阶段和不同视角;其二,结果具有灵活性,不受范围或格式的约束。为推动该任务的研究,我们构建了OpenEPBench,一个开放式未来事件预测数据集。在问题构建方面,我们从七个视角提出问题,包括地点、时间、事件发展、事件结果、事件影响、事件响应及其他,以助力深入分析和理解事件的全面演化过程。在结果构建方面,我们收集包含结果的自由文本作为基准真值(ground truth),以提供语义完整、细节丰富的结果。此外,我们提出了StkFEP,一个利益相关者增强的未来事件预测框架,该框架结合了开放式场景下的事件特征。我们的方法提取事件涉及的利益相关者来扩展问题,以收集多样化的信息;我们还收集与问题相关且相似的历史事件,以揭示潜在的演化模式。实验结果表明,对于现有大语言模型(LLM)而言,在开放式场景下准确预测未来事件是一项具有挑战性的任务。
英文摘要:
Future event prediction (FEP) is a long-standing and crucial task in the world, as understanding the evolution of events enables early risk identification, informed decision-making, and strategic planning. Existing work typically treats event prediction as classification tasks and confines the outcomes of future events to a fixed scope, such as yes/no questions, candidate set, and taxonomy, which is difficult to include all possible outcomes of future events. In this paper, we introduce OpenEP (an Open-Ended Future Event Prediction task), which generates flexible and diverse predictions aligned with real-world scenarios. This is mainly reflected in two aspects: firstly, the predictive questions are diverse, covering different stages of event development and perspectives; secondly, the outcomes are flexible, without constraints on scope or format. To facilitate the study of this task, we construct OpenEPBench, an open-ended future event prediction dataset. For question construction, we pose questions from seven perspectives, including location, time, event development, event outcome, event impact, event response, and other, to facilitate an in-depth analysis and understanding of the comprehensive evolution of events. For outcome construction, we collect free-form text containing the outcomes as ground truth to provide semantically complete and detail-enriched outcomes. Furthermore, we propose StkFEP, a stakeholder-enhanced future event prediction framework, that incorporates event characteristics for open-ended settings. Our method extracts stakeholders involved in events to extend questions to gather diverse information. We also collect historically events that are relevant and similar to the question to reveal potential evolutionary patterns. Experiment results indicate that accurately predicting future events in open-ended settings is challenging for existing LLMs.
机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。