arXivDaily arXiv每日学术速递 周一至周五更新
arXiv 2406.02472cs.CL
收藏

使用大语言模型分析时序复杂事件?一个面向时序、长上下文理解的基准

Analyzing Temporal Complex Events with Large Language Models? A Benchmark towards Temporal, Long Context Understanding

University of California, Los Angeles(加利福尼亚大学洛杉矶分校);Singapore Management University(新加坡管理大学);Fudan University(复旦大学);National University of Singapore(新加坡国立大学)

Zhihan Zhang, Yixin Cao, Chenchen Ye, Yunshan Ma, Lizi Liao, Tat-Seng Chua

AI总结:

本文针对时序复杂事件(TCE)的分析需求,提出利用大语言模型提取事件链,并构建TCELongBench基准评估LLMs在阅读理解、时序排序和未来事件预测任务上的时序与长文本理解能力,实验表明RAG方法与长上下文窗口模型表现相当。

摘要:

随着在线新闻数量不断增加,数字环境正在快速演变,这强调了对复杂事件进行快速且精准分析的需求。我们将由许多新闻文章在较长时期内组成的复杂事件称为时序复杂事件(Temporal Complex Event,TCE)。本文提出了一种使用大语言模型(LLMs)的新方法,以系统地提取和分析TCE内的事件链,其特征在于关键点和时间戳。我们建立了一个名为TCELongBench的基准,以评估LLMs处理时序动态和理解大量文本的能力。该基准包含三个不同的任务——阅读理解、时序排序和未来事件预测。在实验中,我们利用检索增强生成(RAG)方法和具有长上下文窗口的LLMs来处理TCE的冗长新闻文章。我们的研究结果表明,配备合适检索器的模型与利用长上下文窗口的模型表现出相当的性能。

英文摘要:

The digital landscape is rapidly evolving with an ever-increasing volume of online news, emphasizing the need for swift and precise analysis of complex events. We refer to the complex events composed of many news articles over an extended period as Temporal Complex Event (TCE). This paper proposes a novel approach using Large Language Models (LLMs) to systematically extract and analyze the event chain within TCE, characterized by their key points and timestamps. We establish a benchmark, named TCELongBench, to evaluate the proficiency of LLMs in handling temporal dynamics and understanding extensive text. This benchmark encompasses three distinct tasks - reading comprehension, temporal sequencing, and future event forecasting. In the experiment, we leverage retrieval-augmented generation (RAG) method and LLMs with long context window to deal with lengthy news articles of TCE. Our findings indicate that models with suitable retrievers exhibit comparable performance with those utilizing long context window.

机构、总结、中文摘要均由AI分析而来,可能存在错误,请以论文正文为准。

↑